-- Logs begin at Mon 2022-10-17 02:34:03 UTC, end at Mon 2022-10-17 03:31:52 UTC. -- Oct 17 02:58:07.256258 dv-cinder-846977-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 17 02:58:07.264615 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 17 02:58:07.266610 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 17 02:58:07 2022] *** Oct 17 02:58:07.266610 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 17 02:58:07.266610 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 17 02:58:07.266610 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: nodename: dv-cinder-846977-2-iscsi Oct 17 02:58:07.266610 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: machine: x86_64 Oct 17 02:58:07.266610 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: clock source: unix Oct 17 02:58:07.266610 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: pcre jit disabled Oct 17 02:58:07.266610 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: detected number of CPU cores: 4 Oct 17 02:58:07.266610 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: current working directory: / Oct 17 02:58:07.266610 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: detected binary path: /usr/bin/uwsgi-core Oct 17 02:58:07.266610 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: your processes number limit is 47680 Oct 17 02:58:07.267398 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: your memory page size is 4096 bytes Oct 17 02:58:07.267398 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: detected max file descriptor number: 2048 Oct 17 02:58:07.267398 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: lock engine: pthread robust mutexes Oct 17 02:58:07.267398 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: thunder lock: enabled Oct 17 02:58:07.267398 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 17 02:58:07.267398 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 17 02:58:07.305652 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: Python main interpreter initialized at 0x556d890fa9c0 Oct 17 02:58:07.305652 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: python threads support enabled Oct 17 02:58:07.305652 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: your server socket listen backlog is limited to 100 connections Oct 17 02:58:07.305652 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: your mercy for graceful operations on workers is 90 seconds Oct 17 02:58:07.305991 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: mapped 403077 bytes (393 KB) for 2 cores Oct 17 02:58:07.306071 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: *** Operational MODE: preforking *** Oct 17 02:58:07.306340 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: *** uWSGI is running in multiple interpreter mode *** Oct 17 02:58:07.306340 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: spawned uWSGI master process (pid: 81569) Oct 17 02:58:07.306535 dv-cinder-846977-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 17 02:58:07.308225 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: spawned uWSGI worker 1 (pid: 81581, cores: 1) Oct 17 02:58:07.308225 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: spawned uWSGI worker 2 (pid: 81582, cores: 1) Oct 17 02:58:07.309138 dv-cinder-846977-2-iscsi devstack@keystone.service[81569]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 17 02:58:11.592267 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 02:58:11.592267 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: from cryptography.utils import int_from_bytes Oct 17 02:58:11.593061 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 02:58:11.593061 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: from cryptography.utils import int_from_bytes Oct 17 02:58:11.593903 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 02:58:11.593903 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: from cryptography.utils import int_from_bytes Oct 17 02:58:11.594351 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 02:58:11.594351 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: from cryptography.utils import int_from_bytes Oct 17 02:58:11.644169 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 17 02:58:11.644351 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 17 02:58:11.644537 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] command line args: [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 17 02:58:11.645457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 17 02:58:11.645689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ================================================================================ {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 17 02:58:11.645793 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 17 02:58:11.645910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 17 02:58:11.646018 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] admin_token = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.646097 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] command line args: [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 17 02:58:11.646195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] config_dir = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.646352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 17 02:58:11.646435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.646518 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ================================================================================ {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 17 02:58:11.646589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] config_source = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.646724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] debug = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.646800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] admin_token = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.646949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.647259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] config_dir = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.647259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.647453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.647547 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] config_source = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.647638 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.647736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] debug = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.647848 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.647943 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.648061 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.648156 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.648156 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.648346 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.648346 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_config_append = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.648525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.648617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.648721 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.648815 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_dir = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.648914 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_file = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.649013 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.649013 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_config_append = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.649208 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.649286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.649415 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.649501 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_dir = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.649608 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_file = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.649683 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.649747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.649865 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.649969 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.650099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.650177 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.650348 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.650484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.650609 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.650765 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.651006 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.651006 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.651256 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.651380 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.651521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.651664 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.651798 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.651929 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.652054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.652188 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.652317 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.652419 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.652495 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.652576 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.652648 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.652720 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.652720 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.652845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.653835 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] publish_errors = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.653937 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.654015 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.654015 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.654223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.654358 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.654491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] publish_errors = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.654637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.654766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.654887 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.654964 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.654964 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.655120 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] standard_threads = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.655120 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.655245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.655316 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.655387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.655463 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.655534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] standard_threads = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.655604 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] use_journal = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.655604 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] use_json = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.655734 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.655804 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] use_stderr = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.655933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.656008 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] use_syslog = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.656083 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.656149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.656219 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] use_journal = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.656219 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] use_json = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.656350 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.656457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] use_stderr = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.656593 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.656707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] use_syslog = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.656804 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.656934 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 02:58:11.657079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.657202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.657316 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.657423 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.657531 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.657653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.657789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.657930 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.658050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.658170 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.658302 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.658424 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.658537 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.658618 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.658696 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.658766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.658842 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.658930 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.659018 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.659091 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.659186 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.659264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.659436 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.659521 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.659694 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.659780 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.659936 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.660014 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.660194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.660294 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.660995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.661131 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.661264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.661396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.661489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.661560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.661710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.661786 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.661973 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.662050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.662223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.662375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.662502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.662631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.662770 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.662977 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.663110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.663208 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.663287 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.663358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.663435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.663507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.663631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.663771 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.663900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.664027 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.664155 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.664298 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.664408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.664493 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.664566 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.664635 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.664801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.664939 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.665064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.665213 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.665352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.665486 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.665598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.665713 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.665826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.665952 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.666067 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.666186 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.666305 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.666419 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.666541 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.666666 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.666784 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.666987 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.667105 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.667218 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.667339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.667570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.667690 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.667807 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.667924 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.668036 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.668149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.668275 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.668398 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.668514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.668629 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.668743 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.668856 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.668981 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.669107 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.669225 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.669349 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.669463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.669582 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.669699 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.669798 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.669909 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.670010 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.670110 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.670211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.670313 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.670412 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.670504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.670649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.670757 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.670907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.671032 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.671151 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.671265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.671400 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.671524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.671652 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.671773 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.671901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.672012 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.672134 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.672252 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.672398 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.672508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.672594 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.672663 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.672724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.672794 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.672853 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.672911 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.672979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.673045 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.673104 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.673215 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.673314 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.673490 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.673595 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.673725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.673854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.674008 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.674126 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.674256 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.674397 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.674522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.674644 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.674776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.674995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.675124 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.675270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.675381 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.675453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.675513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.675584 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.675656 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.675735 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.675906 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.676055 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.676152 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.676278 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.676409 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.676553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.676670 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.676813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.676893 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.677065 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.677143 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.677314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.677394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.677565 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.677699 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.677826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.677956 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.678102 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.678199 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.678280 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.678423 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.678503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.678681 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.678833 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.678943 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.679018 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.679183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.679261 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.679419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.679507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.679670 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.679796 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.679899 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.680017 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.680149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.680289 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.680419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.680532 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.680646 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.680771 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.680898 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.681026 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.681152 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.681230 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.681327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.681407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.681571 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.681650 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.681807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.external = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.681895 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.682041 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.682127 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.682342 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.682448 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.682581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.682661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.682817 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.password = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.683015 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.683109 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] auth.token = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.683185 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.683336 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.683416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.external = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.683576 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.683656 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.683827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.683941 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.684063 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.684186 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.684312 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.684427 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.password = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.684562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.684670 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] auth.token = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.684804 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.684923 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.685063 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.685195 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.685330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.685466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.685590 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.685672 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.685755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.685854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.685972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.686289 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.686436 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.686569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.686717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.686844 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.686986 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.687060 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.687131 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.687202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.687329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.687458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.687590 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.687720 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.687854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.687977 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.688106 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.688225 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.688340 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.688457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.688598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.688711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.688839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.688958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.689076 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.689194 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.689313 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.689431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.689558 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.689675 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.689798 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.689917 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.690033 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.690149 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.690288 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.690409 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.690510 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.690625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.690740 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.690893 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.691016 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.691134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.691249 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.691365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.691483 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.691621 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.691746 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.691874 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.691994 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.692115 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.692232 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.692366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.692487 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.692604 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.692713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.692874 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.692992 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.693216 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.693336 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.693336 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.693565 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.693676 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.693862 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.693997 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.694112 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.694231 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.694350 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.694471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.694586 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.694675 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.694751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.694838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.694958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.695031 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.695187 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.695312 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.695440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.695557 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.695682 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.695829 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.695961 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.696107 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.696240 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.696355 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.696432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.696504 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.696605 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.696697 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.696860 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.696947 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.697201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.697293 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.697368 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.697450 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.697575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.698539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.698634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.698711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.698823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.699003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.699137 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.699272 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.699403 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.699488 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.699561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.699643 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.699718 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.699790 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.699869 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.699942 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.700014 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.700094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.700167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.700239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.700311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.700383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.700444 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.700512 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.700573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.700632 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.700690 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.700840 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.700982 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.701162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.701343 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.701541 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.701729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.701907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.702112 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.702313 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.702479 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.702621 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.702751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.702899 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.703036 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.703165 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.703291 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.703418 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.703545 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.703671 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.703806 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.703935 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.704060 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.704191 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.704310 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.704446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.704446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.704704 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.704837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.704972 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.705158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.705355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.705520 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.705644 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.705770 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.705896 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.706022 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.706192 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.706405 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.706604 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.706805 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.707059 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.707673 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.707841 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.708047 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.708250 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.708432 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.708562 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.708689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.708811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.708949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.709108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.709314 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.709512 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.709710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.709891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.710085 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.710268 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.710268 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.710600 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.710778 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.710980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.711182 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.711364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.711543 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.711736 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.711916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.712100 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.712293 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.712476 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.712655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.712842 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.713022 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.713203 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.713400 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.713583 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.713762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.713954 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.714136 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.714332 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.714516 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.714693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.714943 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.715120 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.715302 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.715484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.715685 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.715870 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.716051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.716233 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.716403 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.716547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.716758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.716973 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.717174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.717320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.717449 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.717576 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.717722 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.717911 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.717911 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.718279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.718279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.718632 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.718835 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.719050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.719050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.719434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.719434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.719700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.719836 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.719960 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.720982 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.721128 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.721128 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.721499 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.721718 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.721879 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.722005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.722128 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.722262 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] role.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.722386 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.723415 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] role.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.723552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.723745 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.723938 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.724124 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.724402 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.724594 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.724757 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.724885 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.725010 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.725133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.725271 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.725414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.725541 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.725668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.725795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.725927 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.726063 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.726191 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.726316 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] role.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.726442 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.726567 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] role.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.726692 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.726827 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.726976 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.727110 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.727238 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.727365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.727492 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.727629 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.727629 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.727871 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.727998 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.728124 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.728200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.728330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.728454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.728576 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.728708 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.728831 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.728953 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.729091 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.729282 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.729473 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.729672 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.729876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.730054 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.730189 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.730189 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.730421 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.730560 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.730560 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.731015 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.731218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.731366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.731366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.731607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.731771 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.731978 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.732181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.732181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.732536 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.732715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.732896 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.733086 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.733278 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.733458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.733641 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.733834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.734014 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.734206 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.734388 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.734569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.734756 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.734953 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.735171 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.735368 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.735547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.735742 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.735923 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.736106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.736289 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.736484 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.736668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.736844 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.737024 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.737196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.737374 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.737556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.737730 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.737910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.738090 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.738287 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.738492 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.738679 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.738909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.739092 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.739219 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.739382 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.739579 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.739811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.740026 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.740161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.740290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.740419 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.740573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.connection = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.740777 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.740990 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.741178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.741367 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.741558 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.741747 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.741937 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.742141 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.742336 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.742466 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.742592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.742721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.742847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.743931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.743931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.744300 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.744496 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.744703 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.744861 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.744993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.connection = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.745121 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.745258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.745387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.745517 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.745640 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.745769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.745851 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.745923 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.746059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.746189 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.746312 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.746434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.746623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.746822 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.747044 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.747224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.747396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 17 02:58:11.747592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.747592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.747592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.747592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.747592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.747592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.747592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.747592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.747592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.748784 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 02:58:11.748784 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 17 02:58:11.764136 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:11.764421 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:11.764964 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:11.765839 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:11.766080 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:11.766314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:11.773694 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:11.775374 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:11.802835 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:11.803134 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:11.804285 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:11.804525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:11.858837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:11.859105 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:11.909417 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:11.909725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:11.911534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:11.911897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.072161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.072525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.072659 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.072856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.073110 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.073334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.096704 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.096936 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.097163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.097387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.097609 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.098356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.098578 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.100529 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.100765 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.100990 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.101210 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.101429 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.101648 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.109967 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.114799 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.115058 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.115287 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.115511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.115735 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.115960 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.116175 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.123396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.123635 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.123865 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.124094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.124319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.124539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.124766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.129599 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.134203 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.134442 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.134669 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.134921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.135142 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.135358 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.135572 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 02:58:12.141959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.142176 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 02:58:12.142525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.142726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 02:58:12.143026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.143292 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.143557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.143819 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.145831 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.146043 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 02:58:12.146389 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.146590 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 02:58:12.146883 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.147163 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.147427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.147685 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.153433 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.153720 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.153998 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.154286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.161808 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.162449 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.162734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.163111 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.163401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.164739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.165032 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.165322 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.165613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.165901 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.166164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.166437 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.166715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.167019 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.167300 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.167574 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.170319 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.173240 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.173531 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.173820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.174105 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.174391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.174652 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.174949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.175230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.175511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.175789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.176696 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.190729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.192383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.192670 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.197378 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.197669 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.197956 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.202784 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.204292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.204420 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.204580 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.204710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.209993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.210387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.210676 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.210800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.211094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.211389 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.211684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.211974 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.212265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.212620 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.212817 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.217153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.217443 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.221762 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.222063 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.222345 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.225258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.225571 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.225860 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.226152 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.226430 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.228967 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.229258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.233158 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.236643 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.236955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.241350 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.242381 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.242666 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.244621 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.247892 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.248199 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.248491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.248781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.248860 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.249066 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.249361 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.255279 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.255957 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.261368 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.265495 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.265790 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.266067 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.266332 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.266997 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.273770 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.274136 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.274632 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.275321 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.275478 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.275830 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.276338 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.276837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.276989 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.277190 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.280013 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.280307 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.280595 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.280881 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.281284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.293071 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.293365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.293647 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.293935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.294328 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.294616 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.294931 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.299398 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.299692 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.299960 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.307303 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.308970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.310352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.310499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.310770 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.313298 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.313424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.313714 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.313998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.314289 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.314574 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.316250 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.317769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.319395 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.319688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.320091 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.326483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.327979 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.330174 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.330466 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.330751 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.330901 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.331062 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.331311 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.331395 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.331603 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.331684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.331889 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.331982 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.340233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.340535 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.340823 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.347746 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.348604 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.348693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.348943 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.349249 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.349531 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.349811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.350095 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.350386 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.350669 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.350972 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.351269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.351539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.351820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.351910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.352103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.354714 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.356177 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.357704 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.357992 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.358331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.370770 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.373608 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.379443 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.379809 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.380100 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.380388 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.387105 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556d890fa9c0 pid: 81582 (default app) Oct 17 02:58:12.388297 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.388595 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.388888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.390424 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db80302-ade0-46a4-abec-5b1bbd2090ab None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:12.390718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db80302-ade0-46a4-abec-5b1bbd2090ab None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:12.391035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db80302-ade0-46a4-abec-5b1bbd2090ab None None] PATH_INFO: `/v3` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:12.395893 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Mon Oct 17 02:58:12 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:12.396760 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 02:58:12.397055 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.397346 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.397641 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.397916 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.398199 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.398489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.398777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.399064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.399349 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.399643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.399909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.400190 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.400476 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 02:58:12.444826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556d890fa9c0 pid: 81581 (default app) Oct 17 02:58:19.140621 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fd522-31c9-45f2-b4d9-03dc01113bea None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:19.141133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fd522-31c9-45f2-b4d9-03dc01113bea None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:19.141238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fd522-31c9-45f2-b4d9-03dc01113bea None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:19.142497 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 2/2] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 02:58:19.150804 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5f812-de17-4f6b-9fd4-708858842744 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:19.151274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5f812-de17-4f6b-9fd4-708858842744 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:19.151411 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5f812-de17-4f6b-9fd4-708858842744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:19.221449 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: INFO dbcounter [None req-11b5f812-de17-4f6b-9fd4-708858842744 None None] Registered counter for database keystone Oct 17 02:58:19.241222 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG oslo_db.sqlalchemy.engines [None req-11b5f812-de17-4f6b-9fd4-708858842744 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81581) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 17 02:58:19.247318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writer thread running {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 17 02:58:19.424322 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG passlib.handlers.bcrypt [None req-11b5f812-de17-4f6b-9fd4-708858842744 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81581) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 17 02:58:19.424628 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG passlib.handlers.bcrypt [None req-11b5f812-de17-4f6b-9fd4-708858842744 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81581) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 17 02:58:19.482142 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-11b5f812-de17-4f6b-9fd4-708858842744 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:19.483743 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-11b5f812-de17-4f6b-9fd4-708858842744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:19.612085 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1/3] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 02:58:19.901537 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653fd522-31c9-45f2-b4d9-03dc01113bea None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:19.972059 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: INFO dbcounter [None req-653fd522-31c9-45f2-b4d9-03dc01113bea None None] Registered counter for database keystone Oct 17 02:58:19.994026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG oslo_db.sqlalchemy.engines [None req-653fd522-31c9-45f2-b4d9-03dc01113bea None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81582) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 17 02:58:20.000379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writer thread running {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 17 02:58:20.118713 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653fd522-31c9-45f2-b4d9-03dc01113bea None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:20.122992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5b02a-c450-4e55-b12c-5eb5e1d3b2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:20.123639 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5b02a-c450-4e55-b12c-5eb5e1d3b2cf admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:20.123918 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5b02a-c450-4e55-b12c-5eb5e1d3b2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:20.124190 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5b02a-c450-4e55-b12c-5eb5e1d3b2cf admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:20.133103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5b02a-c450-4e55-b12c-5eb5e1d3b2cf admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:20.184957 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: WARNING keystone.common.rbac_enforcer.enforcer [None req-32a5b02a-c450-4e55-b12c-5eb5e1d3b2cf admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 17 02:58:20.185473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5b02a-c450-4e55-b12c-5eb5e1d3b2cf admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:20.195832 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 3/4] 10.222.0.27 () {58 vars in 1324 bytes} [Mon Oct 17 02:58:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 02:58:20.201798 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5f812-de17-4f6b-9fd4-708858842744 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:20.217699 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5f812-de17-4f6b-9fd4-708858842744 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:20.222122 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c9dc5-6c7d-4884-9657-8acbfff6fc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:20.223426 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c9dc5-6c7d-4884-9657-8acbfff6fc15 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:20.223708 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c9dc5-6c7d-4884-9657-8acbfff6fc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:20.223986 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c9dc5-6c7d-4884-9657-8acbfff6fc15 admin admin] PATH_INFO: `/v3/projects/eb6be797d9e4443fad02ceee69bd83a7` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:20.225436 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c9dc5-6c7d-4884-9657-8acbfff6fc15 admin admin] RBAC: Authorizing `identity:get_project(project_id=eb6be797d9e4443fad02ceee69bd83a7)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:20.278832 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4e6c9dc5-6c7d-4884-9657-8acbfff6fc15 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 17 02:58:20.279205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c9dc5-6c7d-4884-9657-8acbfff6fc15 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:20.283809 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 2/5] 10.222.0.27 () {56 vars in 1207 bytes} [Mon Oct 17 02:58:20 2022] GET /identity/v3/projects/eb6be797d9e4443fad02ceee69bd83a7 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:58:22.437881 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a4139-2379-4001-ad15-81db74fa17e1 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:22.438443 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a4139-2379-4001-ad15-81db74fa17e1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:22.438443 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a4139-2379-4001-ad15-81db74fa17e1 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:22.443147 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 4/6] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 02:58:22.448621 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b85dc-3ca5-4c0e-b981-317c9c9a4787 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:22.449061 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b85dc-3ca5-4c0e-b981-317c9c9a4787 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:22.449154 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b85dc-3ca5-4c0e-b981-317c9c9a4787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:22.469981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-929b85dc-3ca5-4c0e-b981-317c9c9a4787 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:22.471551 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-929b85dc-3ca5-4c0e-b981-317c9c9a4787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:22.479727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 3/7] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 17 02:58:22.744564 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a4139-2379-4001-ad15-81db74fa17e1 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:22.757516 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a4139-2379-4001-ad15-81db74fa17e1 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:22.761638 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111729b-0a4e-4fa9-b9c6-8622fe33f804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:22.762233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111729b-0a4e-4fa9-b9c6-8622fe33f804 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:22.762514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111729b-0a4e-4fa9-b9c6-8622fe33f804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:22.762787 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111729b-0a4e-4fa9-b9c6-8622fe33f804 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:22.770342 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111729b-0a4e-4fa9-b9c6-8622fe33f804 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:22.772133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111729b-0a4e-4fa9-b9c6-8622fe33f804 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:22.780680 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 5/8] 10.222.0.27 () {58 vars in 1324 bytes} [Mon Oct 17 02:58:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 02:58:22.785901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b85dc-3ca5-4c0e-b981-317c9c9a4787 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:22.794690 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b85dc-3ca5-4c0e-b981-317c9c9a4787 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:22.798931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300ce74-fc16-46a7-9688-dc0a63175a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:22.799550 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300ce74-fc16-46a7-9688-dc0a63175a07 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:22.799826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300ce74-fc16-46a7-9688-dc0a63175a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:22.800096 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300ce74-fc16-46a7-9688-dc0a63175a07 admin admin] PATH_INFO: `/v3/users/f84d693b9d0643c69861a5541952b43b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:22.802007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300ce74-fc16-46a7-9688-dc0a63175a07 admin admin] RBAC: Authorizing `identity:get_user(user_id=f84d693b9d0643c69861a5541952b43b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:22.803858 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300ce74-fc16-46a7-9688-dc0a63175a07 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:22.805360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 4/9] 10.222.0.27 () {56 vars in 1198 bytes} [Mon Oct 17 02:58:22 2022] GET /identity/v3/users/f84d693b9d0643c69861a5541952b43b => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:25.000246 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d952890-a3e2-4e8a-8fa1-8cca2a94df79 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:25.000784 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d952890-a3e2-4e8a-8fa1-8cca2a94df79 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:25.000784 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d952890-a3e2-4e8a-8fa1-8cca2a94df79 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:25.002183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 6/10] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:25.008329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b7819-f31f-43e3-af07-7426b5c3ff40 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:25.008760 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b7819-f31f-43e3-af07-7426b5c3ff40 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:25.009022 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b7819-f31f-43e3-af07-7426b5c3ff40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:25.027091 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3c0b7819-f31f-43e3-af07-7426b5c3ff40 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:25.028296 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3c0b7819-f31f-43e3-af07-7426b5c3ff40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:25.030984 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f63f3-b036-47d4-8724-1e188620d61b None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:25.031337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f63f3-b036-47d4-8724-1e188620d61b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:25.031682 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f63f3-b036-47d4-8724-1e188620d61b None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:25.032977 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 7/11] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 02:58:25.038179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea560964-85a8-4ce2-8a34-5f8beccf5181 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:25.038462 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea560964-85a8-4ce2-8a34-5f8beccf5181 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:25.038730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea560964-85a8-4ce2-8a34-5f8beccf5181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:25.080114 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG passlib.handlers.bcrypt [None req-ea560964-85a8-4ce2-8a34-5f8beccf5181 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81582) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 17 02:58:25.080507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG passlib.handlers.bcrypt [None req-ea560964-85a8-4ce2-8a34-5f8beccf5181 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81582) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 17 02:58:25.109735 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 5/12] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:25.131550 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ea560964-85a8-4ce2-8a34-5f8beccf5181 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:25.133128 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ea560964-85a8-4ce2-8a34-5f8beccf5181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:25.230538 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 8/13] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:58:25.378299 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b7819-f31f-43e3-af07-7426b5c3ff40 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:25.402748 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:25.482232 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b7819-f31f-43e3-af07-7426b5c3ff40 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:25.494722 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea560964-85a8-4ce2-8a34-5f8beccf5181 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:25.519348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67343d24-adcc-4add-be48-0f90d21c96bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:25.520575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67343d24-adcc-4add-be48-0f90d21c96bd None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:25.520950 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67343d24-adcc-4add-be48-0f90d21c96bd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:25.521114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67343d24-adcc-4add-be48-0f90d21c96bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:25.532544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67343d24-adcc-4add-be48-0f90d21c96bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:58:25.541391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-67343d24-adcc-4add-be48-0f90d21c96bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:58:25.541391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:25.541391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:25.541391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:25.541391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:25.541391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:25.541391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:25.541391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:25.541391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:25.541391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:25.541391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:25.541391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:25.541391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:58:25.541391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:58:25.541391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:58:25.541391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:58:25.542320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:58:25.542320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:58:25.542320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:58:25.542320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:58:25.542320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:25.542320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:25.542320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:25.542320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:25.542320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:25.542320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:25.542320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:25.542320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:25.542320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:25.542320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:25.542320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:25.542320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:25.543223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:25.543223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:25.543223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:25.543223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:25.543223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:58:25.543223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:58:25.543223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:58:25.543223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:58:25.543223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:58:25.543223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:58:25.543223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:58:25.543223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:58:25.543223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 6/14] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 02:58:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:58:25.548256 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67343d24-adcc-4add-be48-0f90d21c96bd None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:25.598574 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea560964-85a8-4ce2-8a34-5f8beccf5181 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:25.600925 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:25.630192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:25.637465 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5cef26-8d66-4063-a739-9e973cbd75f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:25.638556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5cef26-8d66-4063-a739-9e973cbd75f2 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:25.638849 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5cef26-8d66-4063-a739-9e973cbd75f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:25.639157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5cef26-8d66-4063-a739-9e973cbd75f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:25.640126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5cef26-8d66-4063-a739-9e973cbd75f2 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:25.641568 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5cef26-8d66-4063-a739-9e973cbd75f2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:25.649470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67343d24-adcc-4add-be48-0f90d21c96bd None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:25.653016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.sql.core [None req-df5cef26-8d66-4063-a739-9e973cbd75f2 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 02:58:25.653016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 02:58:25.653016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [parameters: {'id': '8c3c852b9e274ff2bd717cc0ca00e38a', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 02:58:25.653016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81582) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 02:58:25.672584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-df5cef26-8d66-4063-a739-9e973cbd75f2 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 02:58:25.672584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:25.672584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 02:58:25.672584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 02:58:25.672584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 02:58:25.672584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 02:58:25.672584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 02:58:25.672584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 02:58:25.672584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 02:58:25.672584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application conn.query(q) Oct 17 02:58:25.672584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 02:58:25.672584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 02:58:25.672584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 02:58:25.672584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result.read() Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 02:58:25.673591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 02:58:25.674529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application next(self.gen) Oct 17 02:58:25.674529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 02:58:25.674529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application yield resource Oct 17 02:58:25.674529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 02:58:25.674529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application next(self.gen) Oct 17 02:58:25.674529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 02:58:25.674529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 02:58:25.674529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 02:58:25.674529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 02:58:25.674529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 02:58:25.674529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise self.value Oct 17 02:58:25.674529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 02:58:25.674529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 02:58:25.674529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 02:58:25.674529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application session.commit() Oct 17 02:58:25.674529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 02:58:25.674529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 02:58:25.675686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 02:58:25.675686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 02:58:25.675686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 02:58:25.675686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.session.flush() Oct 17 02:58:25.675686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 02:58:25.675686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 02:58:25.675686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 02:58:25.675686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 02:58:25.675686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 02:58:25.675686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application compat.raise_( Oct 17 02:58:25.675686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 02:58:25.675686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception Oct 17 02:58:25.675686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 02:58:25.675686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 02:58:25.675686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 02:58:25.675686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 02:58:25.675686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 02:58:25.677525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 02:58:25.677525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 02:58:25.677525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 02:58:25.677525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 02:58:25.677525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 02:58:25.677525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 02:58:25.677525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 02:58:25.677525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 02:58:25.677525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 02:58:25.677525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 02:58:25.677525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 02:58:25.677525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 02:58:25.677525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 02:58:25.677525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 02:58:25.677525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 02:58:25.678487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 02:58:25.678487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception Oct 17 02:58:25.678487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 02:58:25.678487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 02:58:25.678487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 02:58:25.678487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 02:58:25.678487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 02:58:25.678487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 02:58:25.678487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 02:58:25.678487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application conn.query(q) Oct 17 02:58:25.678487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 02:58:25.678487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 02:58:25.678487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 02:58:25.678487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result.read() Oct 17 02:58:25.678487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 02:58:25.678487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 02:58:25.678487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 02:58:25.679719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 02:58:25.679719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 02:58:25.679719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 02:58:25.679719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 02:58:25.679719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 02:58:25.679719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 02:58:25.679719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 02:58:25.679719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [parameters: {'id': '8c3c852b9e274ff2bd717cc0ca00e38a', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 02:58:25.679719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 02:58:25.679719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:58:25.679719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 02:58:25.679719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:58:25.679719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:25.679719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 02:58:25.679719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 02:58:25.680668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 02:58:25.680668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 02:58:25.680668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 17 02:58:25.680668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:58:25.680668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 02:58:25.680668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:58:25.680668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:25.680668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:25.680668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:25.680668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:25.680668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:25.680668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:25.680668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:25.680668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:25.680668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:25.680668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:25.680668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:25.681697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 17 02:58:25.681697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 17 02:58:25.681697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:25.681697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:25.681697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 17 02:58:25.681697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 17 02:58:25.681697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:25.681697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:25.681697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 02:58:25.681697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 02:58:25.681697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 02:58:25.681697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:58:25.681697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 9/15] 10.222.0.27 () {60 vars in 1137 bytes} [Mon Oct 17 02:58:25 2022] POST /identity/v3/domains => generated 190 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:58:25.681697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5cef26-8d66-4063-a739-9e973cbd75f2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:25.689953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972acc4-6cc5-4a8e-80fd-c9f4aabb5621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:25.690573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972acc4-6cc5-4a8e-80fd-c9f4aabb5621 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:25.690857 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972acc4-6cc5-4a8e-80fd-c9f4aabb5621 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:25.691160 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972acc4-6cc5-4a8e-80fd-c9f4aabb5621 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:25.691984 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972acc4-6cc5-4a8e-80fd-c9f4aabb5621 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:25.692453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972acc4-6cc5-4a8e-80fd-c9f4aabb5621 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:25.693864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972acc4-6cc5-4a8e-80fd-c9f4aabb5621 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:25.705733 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 7/16] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:58:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:58:25.712300 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972acc4-6cc5-4a8e-80fd-c9f4aabb5621 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:25.780508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5cef26-8d66-4063-a739-9e973cbd75f2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:25.810366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972acc4-6cc5-4a8e-80fd-c9f4aabb5621 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:25.814541 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef10a2-caee-46a6-bcd5-56c0a0743e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:25.815426 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef10a2-caee-46a6-bcd5-56c0a0743e42 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:25.815710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef10a2-caee-46a6-bcd5-56c0a0743e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:25.815982 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef10a2-caee-46a6-bcd5-56c0a0743e42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:25.834374 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef10a2-caee-46a6-bcd5-56c0a0743e42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:25.835888 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef10a2-caee-46a6-bcd5-56c0a0743e42 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:25.842967 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c03f2-3c0a-4e13-807e-f53fb23ee9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:25.843593 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c03f2-3c0a-4e13-807e-f53fb23ee9b9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:25.843871 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c03f2-3c0a-4e13-807e-f53fb23ee9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:25.844143 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c03f2-3c0a-4e13-807e-f53fb23ee9b9 None admin] PATH_INFO: `/v3/users/f84d693b9d0643c69861a5541952b43b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:25.845279 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c03f2-3c0a-4e13-807e-f53fb23ee9b9 None admin] RBAC: Authorizing `identity:get_user(user_id=f84d693b9d0643c69861a5541952b43b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:25.846716 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c03f2-3c0a-4e13-807e-f53fb23ee9b9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:25.848475 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 8/17] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:58:25 2022] GET /identity/v3/users/f84d693b9d0643c69861a5541952b43b => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:58:25.852178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-2aef10a2-caee-46a6-bcd5-56c0a0743e42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:58:25.852178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:25.852178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:25.852178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:25.852178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:25.852178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:25.852178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:25.852178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:25.852178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:25.852178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:25.852178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:25.852178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:25.852178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:58:25.852178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:58:25.852178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:58:25.852178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:58:25.853241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:25.853241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:25.853241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:25.853241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:25.853241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:25.853241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:25.853241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:25.853241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:25.853241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:25.853241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:25.853241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:25.853241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:25.853241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:25.853241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:25.853241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:25.853241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:25.853241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:58:25.854075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:58:25.854075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:58:25.854075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:58:25.854075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 10/18] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:58:25.854474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c03f2-3c0a-4e13-807e-f53fb23ee9b9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:25.858693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef10a2-caee-46a6-bcd5-56c0a0743e42 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:25.870123 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:25.881604 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:25.954826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c03f2-3c0a-4e13-807e-f53fb23ee9b9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:25.963447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef10a2-caee-46a6-bcd5-56c0a0743e42 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:25.989879 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a8d6c-6dcd-4552-9ea1-67fa15bb980d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:25.990514 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a8d6c-6dcd-4552-9ea1-67fa15bb980d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:25.990798 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a8d6c-6dcd-4552-9ea1-67fa15bb980d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:25.991103 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a8d6c-6dcd-4552-9ea1-67fa15bb980d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:25.992222 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a8d6c-6dcd-4552-9ea1-67fa15bb980d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:25.993683 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a8d6c-6dcd-4552-9ea1-67fa15bb980d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:25.995424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 9/19] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:58:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:58:25.999345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f11d7-369c-45a5-8212-4161a6f2fa73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:26.000038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f11d7-369c-45a5-8212-4161a6f2fa73 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:26.000321 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f11d7-369c-45a5-8212-4161a6f2fa73 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:26.000832 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f11d7-369c-45a5-8212-4161a6f2fa73 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:26.000923 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a8d6c-6dcd-4552-9ea1-67fa15bb980d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:26.001224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f11d7-369c-45a5-8212-4161a6f2fa73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:26.001527 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f11d7-369c-45a5-8212-4161a6f2fa73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:26.003173 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f11d7-369c-45a5-8212-4161a6f2fa73 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:26.021214 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 11/20] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:58:26.085650 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:26.106478 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a8d6c-6dcd-4552-9ea1-67fa15bb980d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:26.141731 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15ae5a-e3e8-4da1-ab9d-9c802a43dd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:26.142405 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15ae5a-e3e8-4da1-ab9d-9c802a43dd99 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:26.142686 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15ae5a-e3e8-4da1-ab9d-9c802a43dd99 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:26.142989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15ae5a-e3e8-4da1-ab9d-9c802a43dd99 None admin] PATH_INFO: `/v3/domains/default/users/f84d693b9d0643c69861a5541952b43b/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:26.144995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15ae5a-e3e8-4da1-ab9d-9c802a43dd99 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f84d693b9d0643c69861a5541952b43b, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:26.146472 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15ae5a-e3e8-4da1-ab9d-9c802a43dd99 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:26.263675 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 10/21] 10.222.0.27 () {58 vars in 1361 bytes} [Mon Oct 17 02:58:25 2022] PUT /identity/v3/domains/default/users/f84d693b9d0643c69861a5541952b43b/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 02:58:28.497714 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddef85-5067-4a8f-822d-2eea9a24362f None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:28.498210 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddef85-5067-4a8f-822d-2eea9a24362f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:28.498281 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddef85-5067-4a8f-822d-2eea9a24362f None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:28.499892 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 12/22] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:28.506824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b587c76-52e5-40ce-9c70-8441b814ba75 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:28.507264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b587c76-52e5-40ce-9c70-8441b814ba75 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:28.507469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b587c76-52e5-40ce-9c70-8441b814ba75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:28.532095 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1b587c76-52e5-40ce-9c70-8441b814ba75 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:28.533581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1b587c76-52e5-40ce-9c70-8441b814ba75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:28.610488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 11/23] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:58:28.902200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dca1e-5b67-44c1-805c-6cbae4166651 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:28.902585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dca1e-5b67-44c1-805c-6cbae4166651 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:28.902755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dca1e-5b67-44c1-805c-6cbae4166651 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:28.904090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 13/24] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:28.919383 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2074f3-4cf2-4bac-a947-e975a7978d50 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:28.919673 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2074f3-4cf2-4bac-a947-e975a7978d50 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:28.919949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2074f3-4cf2-4bac-a947-e975a7978d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:28.943947 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4f2074f3-4cf2-4bac-a947-e975a7978d50 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:28.945145 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4f2074f3-4cf2-4bac-a947-e975a7978d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:29.033782 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:29.040494 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 12/25] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:29.151628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664ebe6-1afd-43e3-91d0-8900bc1bfe38 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:29.151939 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664ebe6-1afd-43e3-91d0-8900bc1bfe38 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:29.152183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664ebe6-1afd-43e3-91d0-8900bc1bfe38 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:29.153678 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 14/26] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:29.156349 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2074f3-4cf2-4bac-a947-e975a7978d50 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:29.160317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653ce50-ade3-461f-9b5a-75898d138ada None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:29.160689 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653ce50-ade3-461f-9b5a-75898d138ada None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:29.160937 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653ce50-ade3-461f-9b5a-75898d138ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:29.192791 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9653ce50-ade3-461f-9b5a-75898d138ada None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:29.194000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9653ce50-ade3-461f-9b5a-75898d138ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:29.323141 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2074f3-4cf2-4bac-a947-e975a7978d50 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:29.333591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 15/27] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:29.335032 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d625e43-5e23-407e-93d4-ae5e1715c864 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:29.335320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d625e43-5e23-407e-93d4-ae5e1715c864 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:29.335596 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d625e43-5e23-407e-93d4-ae5e1715c864 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:29.336965 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 16/28] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:29.348650 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5212d4-acdb-4fff-b0d4-5d047e80e79a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:29.348943 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5212d4-acdb-4fff-b0d4-5d047e80e79a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:29.349305 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5212d4-acdb-4fff-b0d4-5d047e80e79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:29.382488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b40362-3299-40dd-86ef-5573c01df031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:29.383530 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b40362-3299-40dd-86ef-5573c01df031 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:29.384019 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b40362-3299-40dd-86ef-5573c01df031 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:29.384495 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b40362-3299-40dd-86ef-5573c01df031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:29.387860 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6e5212d4-acdb-4fff-b0d4-5d047e80e79a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:29.389052 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6e5212d4-acdb-4fff-b0d4-5d047e80e79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:29.396385 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b40362-3299-40dd-86ef-5573c01df031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:29.397197 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:29.399454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b40362-3299-40dd-86ef-5573c01df031 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:29.428888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-61b40362-3299-40dd-86ef-5573c01df031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:58:29.428888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:29.428888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:29.428888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:29.428888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:29.428888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:29.428888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:29.428888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:29.428888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:29.428888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:29.428888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:29.428888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:29.428888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:58:29.428888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:58:29.428888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:58:29.428888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:58:29.430064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:29.430064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:29.430064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:29.430064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:29.430064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:29.430064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:29.430064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:29.430064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:29.430064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:29.430064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:29.430064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:29.430064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:29.430064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:29.430064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:29.430064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:29.430064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:29.430064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:58:29.431147 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:58:29.431147 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:58:29.431147 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:58:29.436307 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 13/29] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:58:29.437879 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b40362-3299-40dd-86ef-5573c01df031 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:29.519272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:29.530750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 17/30] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:29.532120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5212d4-acdb-4fff-b0d4-5d047e80e79a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:29.567239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b40362-3299-40dd-86ef-5573c01df031 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:29.633000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e3949-b8a4-4a5d-a77b-c38ca73432a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:29.633626 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e3949-b8a4-4a5d-a77b-c38ca73432a5 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:29.633908 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e3949-b8a4-4a5d-a77b-c38ca73432a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:29.634184 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27e3949-b8a4-4a5d-a77b-c38ca73432a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:29.634956 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e3949-b8a4-4a5d-a77b-c38ca73432a5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:29.636468 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27e3949-b8a4-4a5d-a77b-c38ca73432a5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:29.675827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 14/31] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Oct 17 02:58:29 2022] POST /identity/v3/roles => generated 225 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:58:29.697952 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e3949-b8a4-4a5d-a77b-c38ca73432a5 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:29.742713 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5212d4-acdb-4fff-b0d4-5d047e80e79a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:29.801012 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb6003-9bcd-4ba0-800f-d0734fdc25e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:29.801768 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb6003-9bcd-4ba0-800f-d0734fdc25e0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:29.802196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb6003-9bcd-4ba0-800f-d0734fdc25e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:29.802475 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb6003-9bcd-4ba0-800f-d0734fdc25e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:29.803170 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb6003-9bcd-4ba0-800f-d0734fdc25e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:29.803476 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb6003-9bcd-4ba0-800f-d0734fdc25e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:29.804887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb6003-9bcd-4ba0-800f-d0734fdc25e0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:29.827881 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:29.835105 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 18/32] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:29.841092 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb6003-9bcd-4ba0-800f-d0734fdc25e0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:29.906898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27e3949-b8a4-4a5d-a77b-c38ca73432a5 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:29.915735 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:29.937319 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8a18b-8d05-4b75-a734-7492884f9f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:29.938022 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8a18b-8d05-4b75-a734-7492884f9f3e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:29.938310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8a18b-8d05-4b75-a734-7492884f9f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:29.938581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8a18b-8d05-4b75-a734-7492884f9f3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:29.951573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8a18b-8d05-4b75-a734-7492884f9f3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:58:29.957127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-def8a18b-8d05-4b75-a734-7492884f9f3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:58:29.957127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:29.957127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:29.957127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:29.957127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:29.957127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:29.957127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:29.957127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:29.957127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:29.957127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:29.957127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:29.957127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:29.957127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:58:29.957127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:58:29.957127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:58:29.957127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:58:29.958596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:58:29.958596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:58:29.958596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:58:29.958596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:58:29.958596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:29.958596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:29.958596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:29.958596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:29.958596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:29.958596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:29.958596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:29.958596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:29.958596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:29.958596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:29.958596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:29.958596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:29.959424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:29.959424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:29.959424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:29.959424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:29.959424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:58:29.959424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:58:29.959424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:58:29.959424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:58:29.959424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:58:29.959424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:58:29.959424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:58:29.959424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:58:29.960802 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 15/33] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 02:58:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:58:29.965197 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8a18b-8d05-4b75-a734-7492884f9f3e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:30.039751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb6003-9bcd-4ba0-800f-d0734fdc25e0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:30.086859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ba30c-4406-45f2-8735-28cb7e277c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:30.087971 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba30c-4406-45f2-8735-28cb7e277c22 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:30.088238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba30c-4406-45f2-8735-28cb7e277c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:30.089002 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba30c-4406-45f2-8735-28cb7e277c22 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:30.089680 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ba30c-4406-45f2-8735-28cb7e277c22 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:30.091747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ba30c-4406-45f2-8735-28cb7e277c22 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:30.115449 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8a18b-8d05-4b75-a734-7492884f9f3e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:30.149778 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ca053-902e-49df-b103-3d4b3b159d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:30.150422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ca053-902e-49df-b103-3d4b3b159d71 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:30.150709 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ca053-902e-49df-b103-3d4b3b159d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:30.151016 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ca053-902e-49df-b103-3d4b3b159d71 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:30.151692 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ca053-902e-49df-b103-3d4b3b159d71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:30.152001 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ca053-902e-49df-b103-3d4b3b159d71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:30.153419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5ca053-902e-49df-b103-3d4b3b159d71 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:30.162664 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 16/34] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:58:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:58:30.167483 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90a24e-f7ac-4617-a836-2530c206c827 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:30.168133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90a24e-f7ac-4617-a836-2530c206c827 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:30.168398 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90a24e-f7ac-4617-a836-2530c206c827 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:30.170073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 17/35] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:30.171718 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90a24e-f7ac-4617-a836-2530c206c827 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:30.208550 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:30.240040 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 19/36] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Oct 17 02:58:29 2022] POST /identity/v3/projects => generated 301 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:58:30.241056 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ba30c-4406-45f2-8735-28cb7e277c22 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:30.304307 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90a24e-f7ac-4617-a836-2530c206c827 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:30.315455 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:30.347287 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d1511-0d5a-4616-9ada-f9ecb5c69e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:30.348776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d1511-0d5a-4616-9ada-f9ecb5c69e2b None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:30.349023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d1511-0d5a-4616-9ada-f9ecb5c69e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:30.349316 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d1511-0d5a-4616-9ada-f9ecb5c69e2b None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:30.350017 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d1511-0d5a-4616-9ada-f9ecb5c69e2b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:30.351645 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d1511-0d5a-4616-9ada-f9ecb5c69e2b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:30.369455 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 18/37] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Oct 17 02:58:30 2022] POST /identity/v3/roles => generated 219 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:58:30.371103 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cebde-485c-4aee-9159-b91d8ca3d4d6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:30.371514 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cebde-485c-4aee-9159-b91d8ca3d4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:30.371857 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cebde-485c-4aee-9159-b91d8ca3d4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:30.380250 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ba30c-4406-45f2-8735-28cb7e277c22 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:30.396707 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ed3cebde-485c-4aee-9159-b91d8ca3d4d6 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:30.398182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ed3cebde-485c-4aee-9159-b91d8ca3d4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:30.420577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c80ec8-4e50-4af3-9489-953d6a4c3ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:30.421701 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c80ec8-4e50-4af3-9489-953d6a4c3ddd None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:30.422200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c80ec8-4e50-4af3-9489-953d6a4c3ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:30.422686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c80ec8-4e50-4af3-9489-953d6a4c3ddd None admin] PATH_INFO: `/v3/users/f84d693b9d0643c69861a5541952b43b` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:30.424700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c80ec8-4e50-4af3-9489-953d6a4c3ddd None admin] RBAC: Authorizing `identity:get_user(user_id=f84d693b9d0643c69861a5541952b43b)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:30.427061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c80ec8-4e50-4af3-9489-953d6a4c3ddd None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:30.429126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 20/38] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:58:30 2022] GET /identity/v3/users/f84d693b9d0643c69861a5541952b43b => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:30.437552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c80ec8-4e50-4af3-9489-953d6a4c3ddd None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:30.569148 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c80ec8-4e50-4af3-9489-953d6a4c3ddd None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:30.599562 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:30.599748 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 19/39] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:30.612210 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1cd57-6794-49ec-b5cb-2912ef011fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:30.613248 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1cd57-6794-49ec-b5cb-2912ef011fae None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:30.614273 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1cd57-6794-49ec-b5cb-2912ef011fae None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:30.615808 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1cd57-6794-49ec-b5cb-2912ef011fae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:30.617288 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1cd57-6794-49ec-b5cb-2912ef011fae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:30.618998 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1cd57-6794-49ec-b5cb-2912ef011fae None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:30.620790 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 21/40] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:58:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:30.626672 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cebde-485c-4aee-9159-b91d8ca3d4d6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:30.658553 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:30.699338 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f65ef-0de8-4d6a-9b9b-6e5e1991481f None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:30.699638 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f65ef-0de8-4d6a-9b9b-6e5e1991481f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:30.699915 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f65ef-0de8-4d6a-9b9b-6e5e1991481f None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:30.701382 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 22/41] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:30.709322 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436267b6-aad1-4aa1-b30d-c8a02ccf9fe4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:30.709875 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436267b6-aad1-4aa1-b30d-c8a02ccf9fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:30.710804 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436267b6-aad1-4aa1-b30d-c8a02ccf9fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:30.733864 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-436267b6-aad1-4aa1-b30d-c8a02ccf9fe4 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:30.736186 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-436267b6-aad1-4aa1-b30d-c8a02ccf9fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:30.742901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cebde-485c-4aee-9159-b91d8ca3d4d6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:30.785020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180d3dc-becf-4eb5-b0ae-a1e1751a70c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:30.786286 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180d3dc-becf-4eb5-b0ae-a1e1751a70c7 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:30.786657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180d3dc-becf-4eb5-b0ae-a1e1751a70c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:30.787020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180d3dc-becf-4eb5-b0ae-a1e1751a70c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:30.787801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180d3dc-becf-4eb5-b0ae-a1e1751a70c7 None admin] RBAC: Adding query filter params (role.id=ea0fd0c10f2f4e529270edbe3525d4c6, scope.domain.id=default, user.id=f84d693b9d0643c69861a5541952b43b) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:30.788117 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180d3dc-becf-4eb5-b0ae-a1e1751a70c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:30.789570 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a180d3dc-becf-4eb5-b0ae-a1e1751a70c7 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:30.797044 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 20/42] 10.222.0.27 () {56 vars in 1322 bytes} [Mon Oct 17 02:58:30 2022] GET /identity/v3/role_assignments?user.id=f84d693b9d0643c69861a5541952b43b&scope.domain.id=default&role.id=ea0fd0c10f2f4e529270edbe3525d4c6 => generated 530 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:30.817661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 23/43] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:58:30.894774 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180d3dc-becf-4eb5-b0ae-a1e1751a70c7 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:30.991727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:31.005409 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a180d3dc-becf-4eb5-b0ae-a1e1751a70c7 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:31.041354 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d731fc9-1d64-42ce-b225-50a1cb1c3ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:31.041974 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d731fc9-1d64-42ce-b225-50a1cb1c3ee7 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:31.042257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d731fc9-1d64-42ce-b225-50a1cb1c3ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:31.042535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d731fc9-1d64-42ce-b225-50a1cb1c3ee7 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:31.043246 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d731fc9-1d64-42ce-b225-50a1cb1c3ee7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:31.044625 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d731fc9-1d64-42ce-b225-50a1cb1c3ee7 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:31.059085 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 21/44] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Oct 17 02:58:30 2022] POST /identity/v3/roles => generated 223 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:58:31.087084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436267b6-aad1-4aa1-b30d-c8a02ccf9fe4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:31.192410 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436267b6-aad1-4aa1-b30d-c8a02ccf9fe4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:31.211392 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:31.231431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7a88b-bd1c-4a00-b229-8885d853c06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:31.232077 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7a88b-bd1c-4a00-b229-8885d853c06d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:31.232363 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7a88b-bd1c-4a00-b229-8885d853c06d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:31.233317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7a88b-bd1c-4a00-b229-8885d853c06d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:31.233745 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7a88b-bd1c-4a00-b229-8885d853c06d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:31.235232 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7a88b-bd1c-4a00-b229-8885d853c06d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:31.236808 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 24/45] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:58:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:31.243135 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d731fc9-1d64-42ce-b225-50a1cb1c3ee7 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:31.345027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d731fc9-1d64-42ce-b225-50a1cb1c3ee7 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:31.378008 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49dfd0-1294-48e3-b53f-5899a012715c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:31.378619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49dfd0-1294-48e3-b53f-5899a012715c None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:31.378927 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49dfd0-1294-48e3-b53f-5899a012715c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:31.379302 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49dfd0-1294-48e3-b53f-5899a012715c None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:31.380101 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49dfd0-1294-48e3-b53f-5899a012715c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:31.381547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49dfd0-1294-48e3-b53f-5899a012715c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:31.404008 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:31.410340 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 22/46] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Oct 17 02:58:31 2022] POST /identity/v3/projects => generated 312 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:58:33.538745 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37b5a4-6989-44ec-8903-45de2633a8fe None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:33.539268 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37b5a4-6989-44ec-8903-45de2633a8fe None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:33.539268 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b37b5a4-6989-44ec-8903-45de2633a8fe None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:33.540581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 25/47] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:33.545892 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f865f57-4176-496d-b88f-baa8041fec7a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:33.546290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f865f57-4176-496d-b88f-baa8041fec7a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:33.546456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f865f57-4176-496d-b88f-baa8041fec7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:33.564756 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-7f865f57-4176-496d-b88f-baa8041fec7a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:33.565944 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7f865f57-4176-496d-b88f-baa8041fec7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:33.630177 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 23/48] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:33.900104 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37b5a4-6989-44ec-8903-45de2633a8fe None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:33.999500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b37b5a4-6989-44ec-8903-45de2633a8fe None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:34.033065 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a061c4-a58e-41a0-911d-d35b52aa7a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:34.033694 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a061c4-a58e-41a0-911d-d35b52aa7a6e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:34.033981 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a061c4-a58e-41a0-911d-d35b52aa7a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:34.034263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a061c4-a58e-41a0-911d-d35b52aa7a6e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:34.035902 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a061c4-a58e-41a0-911d-d35b52aa7a6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:34.037419 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a061c4-a58e-41a0-911d-d35b52aa7a6e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:34.038965 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 26/49] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:58:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:34.044441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f865f57-4176-496d-b88f-baa8041fec7a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:34.142001 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f865f57-4176-496d-b88f-baa8041fec7a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:34.170141 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:34.176785 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbcdf5-89f3-4129-be59-42d8cc5e7029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:34.177396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbcdf5-89f3-4129-be59-42d8cc5e7029 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:34.177683 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbcdf5-89f3-4129-be59-42d8cc5e7029 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:34.177961 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbcdf5-89f3-4129-be59-42d8cc5e7029 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:34.178617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbcdf5-89f3-4129-be59-42d8cc5e7029 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:34.180132 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbcdf5-89f3-4129-be59-42d8cc5e7029 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:34.204969 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 24/50] 10.222.0.27 () {60 vars in 1140 bytes} [Mon Oct 17 02:58:34 2022] POST /identity/v3/projects => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:58:36.330270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a569c-341f-442c-a73f-48dd532ff2cb None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:36.330726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a569c-341f-442c-a73f-48dd532ff2cb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:36.330830 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a569c-341f-442c-a73f-48dd532ff2cb None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:36.332266 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 27/51] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:36.337856 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518c212-48dd-4e0d-987a-1d8f810bd215 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:36.338217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518c212-48dd-4e0d-987a-1d8f810bd215 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:36.338417 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518c212-48dd-4e0d-987a-1d8f810bd215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:36.357790 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5518c212-48dd-4e0d-987a-1d8f810bd215 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:36.359013 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5518c212-48dd-4e0d-987a-1d8f810bd215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:36.424391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 25/52] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:36.696353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a569c-341f-442c-a73f-48dd532ff2cb None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:36.751457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:36.797020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a569c-341f-442c-a73f-48dd532ff2cb None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:36.828841 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c8fb3-e485-4100-baa8-b4ede53fdf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:36.829459 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c8fb3-e485-4100-baa8-b4ede53fdf3e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:36.829742 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c8fb3-e485-4100-baa8-b4ede53fdf3e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:36.830014 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c8fb3-e485-4100-baa8-b4ede53fdf3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:36.831125 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c8fb3-e485-4100-baa8-b4ede53fdf3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:36.833094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c8fb3-e485-4100-baa8-b4ede53fdf3e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:36.834615 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 28/53] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:58:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:36.840017 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518c212-48dd-4e0d-987a-1d8f810bd215 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:36.935752 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518c212-48dd-4e0d-987a-1d8f810bd215 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:36.948239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:36.969334 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12dadf-48b3-4b59-8bf3-62b4d3c30899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:36.969934 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12dadf-48b3-4b59-8bf3-62b4d3c30899 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:36.970214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12dadf-48b3-4b59-8bf3-62b4d3c30899 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:36.970483 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12dadf-48b3-4b59-8bf3-62b4d3c30899 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:36.971162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12dadf-48b3-4b59-8bf3-62b4d3c30899 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:36.973464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12dadf-48b3-4b59-8bf3-62b4d3c30899 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:36.998437 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 26/54] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 02:58:36 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:58:40.364619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384948f-d730-4754-bf92-b67f26c97ada None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:40.364619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384948f-d730-4754-bf92-b67f26c97ada None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:40.364619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384948f-d730-4754-bf92-b67f26c97ada None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:40.364619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 29/55] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:40.369809 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b90d31-8839-435a-aab7-84daa5002d51 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:40.370613 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b90d31-8839-435a-aab7-84daa5002d51 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:40.371137 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b90d31-8839-435a-aab7-84daa5002d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:40.387513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a920e94-38d9-4ab6-a883-4dab021d2ce6 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:40.389034 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a920e94-38d9-4ab6-a883-4dab021d2ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:40.389526 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a920e94-38d9-4ab6-a883-4dab021d2ce6 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:40.391141 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 30/56] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 02:58:40.403019 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a6b90d31-8839-435a-aab7-84daa5002d51 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:40.403893 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6a87c-026d-496e-ba08-c0ec966eaece None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:40.404243 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a6b90d31-8839-435a-aab7-84daa5002d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:40.404652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6a87c-026d-496e-ba08-c0ec966eaece None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:40.405133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6a87c-026d-496e-ba08-c0ec966eaece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:40.443418 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-96a6a87c-026d-496e-ba08-c0ec966eaece None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:40.445044 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-96a6a87c-026d-496e-ba08-c0ec966eaece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:40.511871 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 27/57] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:40.514197 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f954a9-198f-4caa-a1f2-bf9f8878c23d None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:40.514698 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f954a9-198f-4caa-a1f2-bf9f8878c23d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:40.515463 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f954a9-198f-4caa-a1f2-bf9f8878c23d None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:40.517576 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 28/58] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:40.523874 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40777b0-5f06-4709-95ed-3a214331cc78 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:40.524419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40777b0-5f06-4709-95ed-3a214331cc78 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:40.524940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40777b0-5f06-4709-95ed-3a214331cc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:40.546997 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 31/59] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 17 02:58:40.567340 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f40777b0-5f06-4709-95ed-3a214331cc78 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:40.568607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f40777b0-5f06-4709-95ed-3a214331cc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:40.668179 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 29/60] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:58:40.680359 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3776e49-cce2-4376-bb54-527fc5062634 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:40.680659 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3776e49-cce2-4376-bb54-527fc5062634 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:40.680945 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3776e49-cce2-4376-bb54-527fc5062634 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:40.682300 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 32/61] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:40.696380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8ff10-efa7-4dac-b688-f5570c64a0a7 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:40.696674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8ff10-efa7-4dac-b688-f5570c64a0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:40.696960 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8ff10-efa7-4dac-b688-f5570c64a0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:40.718965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-bbd8ff10-efa7-4dac-b688-f5570c64a0a7 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:40.720415 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bbd8ff10-efa7-4dac-b688-f5570c64a0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:40.748378 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:40.815514 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 30/62] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:40.826261 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3776e49-cce2-4376-bb54-527fc5062634 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:40.908933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:40.936962 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8ff10-efa7-4dac-b688-f5570c64a0a7 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:40.972569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3776e49-cce2-4376-bb54-527fc5062634 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:41.019675 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869181a-f61b-4c6a-9c69-ba788d1f3ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:41.021435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869181a-f61b-4c6a-9c69-ba788d1f3ffe None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:41.021728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869181a-f61b-4c6a-9c69-ba788d1f3ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:41.022011 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869181a-f61b-4c6a-9c69-ba788d1f3ffe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:41.037768 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869181a-f61b-4c6a-9c69-ba788d1f3ffe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:58:41.044760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d869181a-f61b-4c6a-9c69-ba788d1f3ffe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:58:41.044760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:41.044760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:41.044760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:41.044760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:41.044760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:41.044760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:41.044760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:41.044760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:41.044760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:41.044760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:41.044760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:41.044760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:58:41.044760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:58:41.044760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:58:41.044760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:58:41.045693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:58:41.045693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:58:41.045693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:58:41.045693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:58:41.045693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:41.045693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:41.045693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:41.045693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:41.045693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:41.045693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:41.045693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:41.045693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:41.045693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:41.045693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:41.045693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:41.045693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:41.046463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:41.046463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:41.046463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:41.046463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:41.046463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:58:41.046463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:58:41.046463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:58:41.046463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:58:41.046463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:58:41.046463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:58:41.046463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:58:41.046463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:58:41.048983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 33/63] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 02:58:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:58:41.055158 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869181a-f61b-4c6a-9c69-ba788d1f3ffe None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:41.076363 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8ff10-efa7-4dac-b688-f5570c64a0a7 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:41.124653 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b86214-4e01-4040-81b7-8734b8dc6284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:41.125307 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b86214-4e01-4040-81b7-8734b8dc6284 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:41.125591 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b86214-4e01-4040-81b7-8734b8dc6284 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:41.125868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b86214-4e01-4040-81b7-8734b8dc6284 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:41.142271 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b86214-4e01-4040-81b7-8734b8dc6284 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:58:41.148020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-f5b86214-4e01-4040-81b7-8734b8dc6284 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:58:41.148020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:41.148020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:41.148020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:41.148020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:41.148020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:41.148020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:41.148020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:41.148020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:41.148020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:41.148020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:41.148020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:41.148020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:58:41.148020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:58:41.148020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:58:41.148020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:58:41.148888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:58:41.148888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:58:41.148888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:58:41.148888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:58:41.148888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:41.148888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:41.148888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:41.148888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:41.148888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:41.148888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:41.148888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:41.148888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:41.148888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:41.148888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:41.148888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:41.148888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:41.149647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:41.149647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:41.149647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:41.149647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:41.149647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:58:41.149647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:58:41.149647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:58:41.149647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:58:41.149647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:58:41.149647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:58:41.149647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:58:41.149647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:58:41.149647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 31/64] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:58:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:58:41.150335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b86214-4e01-4040-81b7-8734b8dc6284 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:41.171251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:41.174828 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869181a-f61b-4c6a-9c69-ba788d1f3ffe None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:41.188169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:41.209777 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba98d6-7338-4f4e-8cb8-12fd8ed22535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:41.210409 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba98d6-7338-4f4e-8cb8-12fd8ed22535 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:41.210694 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba98d6-7338-4f4e-8cb8-12fd8ed22535 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:41.211002 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba98d6-7338-4f4e-8cb8-12fd8ed22535 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:41.221681 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba98d6-7338-4f4e-8cb8-12fd8ed22535 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:58:41.227355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-07ba98d6-7338-4f4e-8cb8-12fd8ed22535 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:58:41.227355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:41.227355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:41.227355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:41.227355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:41.227355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:41.227355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:41.227355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:41.227355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:41.227355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:41.227355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:41.227355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:41.227355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:58:41.227355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:58:41.227355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:58:41.227355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:58:41.228377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:58:41.228377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:58:41.228377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:58:41.228377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:58:41.228377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:41.228377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:41.228377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:41.228377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:41.228377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:41.228377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:41.228377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:41.228377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:41.228377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:41.228377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:41.228377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:41.228377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:41.229168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:41.229168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:41.229168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:41.229168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:41.229168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:58:41.229168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:58:41.229168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:58:41.229168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:58:41.229168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:58:41.229168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:58:41.229168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:58:41.229168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:58:41.229168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 34/65] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Oct 17 02:58:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:58:41.229801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96ceed-457a-4dce-89ba-000cd77e0c43 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:41.230257 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96ceed-457a-4dce-89ba-000cd77e0c43 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:41.230257 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96ceed-457a-4dce-89ba-000cd77e0c43 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:41.231572 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 35/66] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:41.235335 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e96ceed-457a-4dce-89ba-000cd77e0c43 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:41.258812 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b86214-4e01-4040-81b7-8734b8dc6284 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:41.292793 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b606d-e3e9-4086-bb3c-f3bc384f3f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:41.293408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b606d-e3e9-4086-bb3c-f3bc384f3f08 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:41.293719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b606d-e3e9-4086-bb3c-f3bc384f3f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:41.293984 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b606d-e3e9-4086-bb3c-f3bc384f3f08 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:41.294661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b606d-e3e9-4086-bb3c-f3bc384f3f08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:41.295005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b606d-e3e9-4086-bb3c-f3bc384f3f08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:41.296410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b606d-e3e9-4086-bb3c-f3bc384f3f08 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:41.304419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 32/67] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:58:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:41.308979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b606d-e3e9-4086-bb3c-f3bc384f3f08 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:41.341279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e96ceed-457a-4dce-89ba-000cd77e0c43 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:41.373742 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc634824-bf61-469f-a666-1b291686c361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:41.375091 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc634824-bf61-469f-a666-1b291686c361 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:41.375379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc634824-bf61-469f-a666-1b291686c361 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:41.375656 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc634824-bf61-469f-a666-1b291686c361 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:41.376316 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc634824-bf61-469f-a666-1b291686c361 None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:41.376622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc634824-bf61-469f-a666-1b291686c361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:41.378268 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc634824-bf61-469f-a666-1b291686c361 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:41.385489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:41.406031 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b606d-e3e9-4086-bb3c-f3bc384f3f08 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:41.442855 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f9def-d6d9-493e-9a1a-d665b4e82ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:41.443560 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f9def-d6d9-493e-9a1a-d665b4e82ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:41.443843 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f9def-d6d9-493e-9a1a-d665b4e82ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:41.444120 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f9def-d6d9-493e-9a1a-d665b4e82ec0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:41.455489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f9def-d6d9-493e-9a1a-d665b4e82ec0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:58:41.461456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-f80f9def-d6d9-493e-9a1a-d665b4e82ec0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:58:41.461456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:41.461456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:41.461456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:41.461456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:41.461456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:41.461456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:41.461456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:41.461456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:41.461456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:41.461456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:41.461456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:41.461456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:58:41.461456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:58:41.461456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:58:41.461456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:58:41.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:58:41.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:58:41.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:58:41.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:58:41.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:41.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:41.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:41.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:41.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:41.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:41.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:41.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:41.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:41.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:41.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:41.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:41.463422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:41.463422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:41.463422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:41.463422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:41.463422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:58:41.463422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:58:41.463422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:58:41.463422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:58:41.463422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:58:41.463422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:58:41.463422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:58:41.463422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:58:41.463422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 33/68] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:58:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:58:41.464070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f9def-d6d9-493e-9a1a-d665b4e82ec0 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:41.495862 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 36/69] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:58:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:41.497938 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80586637-092b-4e43-94d7-043fd31d3c47 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:41.498236 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80586637-092b-4e43-94d7-043fd31d3c47 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:41.498519 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80586637-092b-4e43-94d7-043fd31d3c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:41.516382 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-80586637-092b-4e43-94d7-043fd31d3c47 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:41.517604 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-80586637-092b-4e43-94d7-043fd31d3c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:41.544701 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:41.568211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f9def-d6d9-493e-9a1a-d665b4e82ec0 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:41.584426 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 37/70] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:41.585263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80586637-092b-4e43-94d7-043fd31d3c47 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:41.617694 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485a5d4-a36e-417b-9dd6-7891027b9def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:41.618244 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485a5d4-a36e-417b-9dd6-7891027b9def None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:41.618557 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485a5d4-a36e-417b-9dd6-7891027b9def None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:41.618813 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485a5d4-a36e-417b-9dd6-7891027b9def None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:41.619517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485a5d4-a36e-417b-9dd6-7891027b9def None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:41.619822 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485a5d4-a36e-417b-9dd6-7891027b9def None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:41.621230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485a5d4-a36e-417b-9dd6-7891027b9def None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:41.630269 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 34/71] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:58:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:41.631168 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485a5d4-a36e-417b-9dd6-7891027b9def None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:41.633830 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:41.693073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80586637-092b-4e43-94d7-043fd31d3c47 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:41.729975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d2691-a752-4410-b1e4-e564181d77d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:41.730698 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d2691-a752-4410-b1e4-e564181d77d8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:41.731083 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d2691-a752-4410-b1e4-e564181d77d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:41.731381 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d2691-a752-4410-b1e4-e564181d77d8 None admin] PATH_INFO: `/v3/users/f84d693b9d0643c69861a5541952b43b` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:41.732488 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d2691-a752-4410-b1e4-e564181d77d8 None admin] RBAC: Authorizing `identity:get_user(user_id=f84d693b9d0643c69861a5541952b43b)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:41.733924 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d2691-a752-4410-b1e4-e564181d77d8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:41.735712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 38/72] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:58:41 2022] GET /identity/v3/users/f84d693b9d0643c69861a5541952b43b => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:41.736273 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d2691-a752-4410-b1e4-e564181d77d8 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:41.744742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485a5d4-a36e-417b-9dd6-7891027b9def None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:41.768480 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:41.783575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068a348-5099-4841-ba9f-c38839a02c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:41.784197 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068a348-5099-4841-ba9f-c38839a02c89 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:41.784479 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068a348-5099-4841-ba9f-c38839a02c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:41.784751 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068a348-5099-4841-ba9f-c38839a02c89 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:41.786146 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a068a348-5099-4841-ba9f-c38839a02c89 None admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:41.786469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a068a348-5099-4841-ba9f-c38839a02c89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:41.787908 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a068a348-5099-4841-ba9f-c38839a02c89 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:41.800638 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 35/73] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:58:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:41.800638 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068a348-5099-4841-ba9f-c38839a02c89 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:41.848908 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d2691-a752-4410-b1e4-e564181d77d8 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:41.861234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:41.884587 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2376be-a65a-4aa5-a9eb-06d9c79ef1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:41.885153 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2376be-a65a-4aa5-a9eb-06d9c79ef1e3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:41.885436 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2376be-a65a-4aa5-a9eb-06d9c79ef1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:41.885716 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2376be-a65a-4aa5-a9eb-06d9c79ef1e3 None admin] PATH_INFO: `/v3/users/f5d5caad74e9468ca5bba4cd444b7404` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:41.901181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2376be-a65a-4aa5-a9eb-06d9c79ef1e3 None admin] RBAC: Authorizing `identity:get_user(user_id=f5d5caad74e9468ca5bba4cd444b7404)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:41.901309 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068a348-5099-4841-ba9f-c38839a02c89 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:41.902631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2376be-a65a-4aa5-a9eb-06d9c79ef1e3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:41.904275 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 39/74] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:58:41 2022] GET /identity/v3/users/f5d5caad74e9468ca5bba4cd444b7404 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:41.905049 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2376be-a65a-4aa5-a9eb-06d9c79ef1e3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:41.933637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf658e-c272-4540-8c5a-3e88b77048f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:41.934288 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf658e-c272-4540-8c5a-3e88b77048f1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:41.934569 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf658e-c272-4540-8c5a-3e88b77048f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:41.934843 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf658e-c272-4540-8c5a-3e88b77048f1 None admin] PATH_INFO: `/v3/users/f5d5caad74e9468ca5bba4cd444b7404` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:41.935991 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf658e-c272-4540-8c5a-3e88b77048f1 None admin] RBAC: Authorizing `identity:get_user(user_id=f5d5caad74e9468ca5bba4cd444b7404)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:41.937446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf658e-c272-4540-8c5a-3e88b77048f1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:41.939032 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 36/75] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:58:41 2022] GET /identity/v3/users/f5d5caad74e9468ca5bba4cd444b7404 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:41.939787 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf658e-c272-4540-8c5a-3e88b77048f1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:41.998554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:42.009286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2376be-a65a-4aa5-a9eb-06d9c79ef1e3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:42.038563 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf658e-c272-4540-8c5a-3e88b77048f1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:42.043167 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7b587-1dcf-4047-991c-eaeb7ea9d90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:42.043797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7b587-1dcf-4047-991c-eaeb7ea9d90c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:42.044079 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7b587-1dcf-4047-991c-eaeb7ea9d90c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:42.044357 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7b587-1dcf-4047-991c-eaeb7ea9d90c None admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:42.045479 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7b587-1dcf-4047-991c-eaeb7ea9d90c None admin] RBAC: Authorizing `identity:get_project(project_id=f64aa9442bc9483987f3c997119762af)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:42.046979 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7b587-1dcf-4047-991c-eaeb7ea9d90c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:42.048604 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 40/76] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:58:41 2022] GET /identity/v3/projects/f64aa9442bc9483987f3c997119762af => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:42.049464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7b587-1dcf-4047-991c-eaeb7ea9d90c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:42.067283 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:42.072406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c9ce3-4fb1-4b7c-b199-d4795ce7869f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:42.073047 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c9ce3-4fb1-4b7c-b199-d4795ce7869f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:42.073335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c9ce3-4fb1-4b7c-b199-d4795ce7869f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:42.073625 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c9ce3-4fb1-4b7c-b199-d4795ce7869f None admin] PATH_INFO: `/v3/users/f5d5caad74e9468ca5bba4cd444b7404` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:42.074764 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c9ce3-4fb1-4b7c-b199-d4795ce7869f None admin] RBAC: Authorizing `identity:get_user(user_id=f5d5caad74e9468ca5bba4cd444b7404)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:42.076254 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c9ce3-4fb1-4b7c-b199-d4795ce7869f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:42.077838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 37/77] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:58:41 2022] GET /identity/v3/users/f5d5caad74e9468ca5bba4cd444b7404 => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:42.078599 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c9ce3-4fb1-4b7c-b199-d4795ce7869f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:42.148764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7b587-1dcf-4047-991c-eaeb7ea9d90c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:42.178342 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c9ce3-4fb1-4b7c-b199-d4795ce7869f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:42.183222 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52cd72-f7a6-43df-8f27-b66d9cde6b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:42.183849 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52cd72-f7a6-43df-8f27-b66d9cde6b46 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:42.184128 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52cd72-f7a6-43df-8f27-b66d9cde6b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:42.184398 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52cd72-f7a6-43df-8f27-b66d9cde6b46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:42.185511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52cd72-f7a6-43df-8f27-b66d9cde6b46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:42.186973 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52cd72-f7a6-43df-8f27-b66d9cde6b46 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:42.188712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 41/78] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:58:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:42.189267 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52cd72-f7a6-43df-8f27-b66d9cde6b46 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:42.212141 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2f067-4754-4aa1-8380-238259b996f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:42.212890 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2f067-4754-4aa1-8380-238259b996f4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:42.213215 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2f067-4754-4aa1-8380-238259b996f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:42.213619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2f067-4754-4aa1-8380-238259b996f4 None admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:42.214997 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2f067-4754-4aa1-8380-238259b996f4 None admin] RBAC: Authorizing `identity:get_project(project_id=f64aa9442bc9483987f3c997119762af)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:42.216462 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2f067-4754-4aa1-8380-238259b996f4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:42.218107 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 38/79] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:58:42 2022] GET /identity/v3/projects/f64aa9442bc9483987f3c997119762af => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:42.219039 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2f067-4754-4aa1-8380-238259b996f4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:42.219141 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:42.291739 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:42.292507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52cd72-f7a6-43df-8f27-b66d9cde6b46 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:42.325071 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c5084-f804-4cff-8b1a-672a026e2006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:42.325241 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2f067-4754-4aa1-8380-238259b996f4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:42.325714 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c5084-f804-4cff-8b1a-672a026e2006 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:42.325993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c5084-f804-4cff-8b1a-672a026e2006 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:42.326268 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c5084-f804-4cff-8b1a-672a026e2006 None admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:42.327426 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c5084-f804-4cff-8b1a-672a026e2006 None admin] RBAC: Authorizing `identity:get_project(project_id=f64aa9442bc9483987f3c997119762af)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:42.328868 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c5084-f804-4cff-8b1a-672a026e2006 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:42.330486 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 42/80] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:58:42 2022] GET /identity/v3/projects/f64aa9442bc9483987f3c997119762af => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:42.335871 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c5084-f804-4cff-8b1a-672a026e2006 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:42.358697 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c11f8-d846-410d-bd5a-55d290e506cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:42.359394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c11f8-d846-410d-bd5a-55d290e506cd None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:42.359678 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c11f8-d846-410d-bd5a-55d290e506cd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:42.359977 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c11f8-d846-410d-bd5a-55d290e506cd None admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af/users/f84d693b9d0643c69861a5541952b43b/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:42.361944 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c11f8-d846-410d-bd5a-55d290e506cd None admin] RBAC: Authorizing `identity:create_grant(project_id=f64aa9442bc9483987f3c997119762af, user_id=f84d693b9d0643c69861a5541952b43b, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:42.363433 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c11f8-d846-410d-bd5a-55d290e506cd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:42.376841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 39/81] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 02:58:42 2022] PUT /identity/v3/projects/f64aa9442bc9483987f3c997119762af/users/f84d693b9d0643c69861a5541952b43b/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:58:42.377790 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c11f8-d846-410d-bd5a-55d290e506cd None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:42.450182 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:42.458934 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c5084-f804-4cff-8b1a-672a026e2006 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:42.493309 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c11f8-d846-410d-bd5a-55d290e506cd None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:42.498197 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cf339-702e-470c-807d-d4944a4f897d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:42.499094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cf339-702e-470c-807d-d4944a4f897d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:42.499528 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cf339-702e-470c-807d-d4944a4f897d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:42.499979 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cf339-702e-470c-807d-d4944a4f897d None admin] PATH_INFO: `/v3/projects/b5cfbc27829541e280a643a64f865804` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:42.501474 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2cf339-702e-470c-807d-d4944a4f897d None admin] RBAC: Authorizing `identity:get_project(project_id=b5cfbc27829541e280a643a64f865804)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:42.503559 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2cf339-702e-470c-807d-d4944a4f897d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:42.506011 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 43/82] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:58:42 2022] GET /identity/v3/projects/b5cfbc27829541e280a643a64f865804 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:42.507034 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cf339-702e-470c-807d-d4944a4f897d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:42.527276 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:42.537873 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6721e-3d4b-4d58-87dd-88d6fd05e832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:42.538507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6721e-3d4b-4d58-87dd-88d6fd05e832 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:42.538789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6721e-3d4b-4d58-87dd-88d6fd05e832 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:42.539089 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6721e-3d4b-4d58-87dd-88d6fd05e832 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:42.539785 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e6721e-3d4b-4d58-87dd-88d6fd05e832 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:42.541269 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e6721e-3d4b-4d58-87dd-88d6fd05e832 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:42.565713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 40/83] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 02:58:42 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:58:42.572030 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6721e-3d4b-4d58-87dd-88d6fd05e832 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:42.639672 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cf339-702e-470c-807d-d4944a4f897d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:42.688194 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0988e-97cd-41b7-b49d-caeebb1b06eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:42.688910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0988e-97cd-41b7-b49d-caeebb1b06eb None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:42.689502 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0988e-97cd-41b7-b49d-caeebb1b06eb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:42.689785 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0988e-97cd-41b7-b49d-caeebb1b06eb None admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af/users/f5d5caad74e9468ca5bba4cd444b7404/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:42.692638 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0988e-97cd-41b7-b49d-caeebb1b06eb None admin] RBAC: Authorizing `identity:create_grant(project_id=f64aa9442bc9483987f3c997119762af, user_id=f5d5caad74e9468ca5bba4cd444b7404, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:42.694843 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0988e-97cd-41b7-b49d-caeebb1b06eb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:42.710471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6721e-3d4b-4d58-87dd-88d6fd05e832 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:42.719494 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 44/84] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 02:58:42 2022] PUT /identity/v3/projects/f64aa9442bc9483987f3c997119762af/users/f5d5caad74e9468ca5bba4cd444b7404/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:58:42.720815 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0988e-97cd-41b7-b49d-caeebb1b06eb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:42.751128 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838cab1-a88c-4468-ad80-692e7953fc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:42.752042 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838cab1-a88c-4468-ad80-692e7953fc77 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:42.752217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838cab1-a88c-4468-ad80-692e7953fc77 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:42.752507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838cab1-a88c-4468-ad80-692e7953fc77 None admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af/users/f5d5caad74e9468ca5bba4cd444b7404/roles/baf207ce98e448f1a2e10c3310a0e33b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:42.755764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:42.755914 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838cab1-a88c-4468-ad80-692e7953fc77 None admin] RBAC: Authorizing `identity:create_grant(project_id=f64aa9442bc9483987f3c997119762af, user_id=f5d5caad74e9468ca5bba4cd444b7404, role_id=baf207ce98e448f1a2e10c3310a0e33b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:42.757425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838cab1-a88c-4468-ad80-692e7953fc77 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:42.794195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 41/85] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 02:58:42 2022] PUT /identity/v3/projects/f64aa9442bc9483987f3c997119762af/users/f5d5caad74e9468ca5bba4cd444b7404/roles/baf207ce98e448f1a2e10c3310a0e33b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:58:42.861758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0988e-97cd-41b7-b49d-caeebb1b06eb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:42.906959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f9128-1f1e-4cab-994a-8b08c4a1ffc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:42.907667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f9128-1f1e-4cab-994a-8b08c4a1ffc1 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:42.907954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f9128-1f1e-4cab-994a-8b08c4a1ffc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:42.908236 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f9128-1f1e-4cab-994a-8b08c4a1ffc1 None admin] PATH_INFO: `/v3/projects/b5cfbc27829541e280a643a64f865804/users/f5d5caad74e9468ca5bba4cd444b7404/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:42.915915 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f9128-1f1e-4cab-994a-8b08c4a1ffc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5cfbc27829541e280a643a64f865804, user_id=f5d5caad74e9468ca5bba4cd444b7404, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:42.917472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f9128-1f1e-4cab-994a-8b08c4a1ffc1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:42.938403 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 45/86] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 02:58:42 2022] PUT /identity/v3/projects/b5cfbc27829541e280a643a64f865804/users/f5d5caad74e9468ca5bba4cd444b7404/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:58:44.903146 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741186b-26d7-4440-8ce7-e517f30b90c1 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:44.903665 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741186b-26d7-4440-8ce7-e517f30b90c1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:44.903737 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741186b-26d7-4440-8ce7-e517f30b90c1 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:44.905193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 42/87] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:44.928953 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc0212-adc6-405d-afc5-ee72e11b5e49 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:44.929443 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc0212-adc6-405d-afc5-ee72e11b5e49 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:44.929514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc0212-adc6-405d-afc5-ee72e11b5e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:44.955888 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2dfc0212-adc6-405d-afc5-ee72e11b5e49 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:44.957087 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2dfc0212-adc6-405d-afc5-ee72e11b5e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:45.029052 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 46/88] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:58:45.350996 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741186b-26d7-4440-8ce7-e517f30b90c1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:45.408978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686d696-b1db-4a59-a4ec-1b335c119e8d None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:45.409625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686d696-b1db-4a59-a4ec-1b335c119e8d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:45.409792 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:45.410250 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686d696-b1db-4a59-a4ec-1b335c119e8d None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:45.412681 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 47/89] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:45.429264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e490eed-519c-415a-bfc2-402563449f66 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:45.429845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e490eed-519c-415a-bfc2-402563449f66 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:45.430350 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e490eed-519c-415a-bfc2-402563449f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:45.455260 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9e490eed-519c-415a-bfc2-402563449f66 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:45.459414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9e490eed-519c-415a-bfc2-402563449f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:45.474949 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:45.519016 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741186b-26d7-4440-8ce7-e517f30b90c1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:45.585686 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57954c6-6ba2-44e7-82a5-d058cb54ec30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:45.588258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57954c6-6ba2-44e7-82a5-d058cb54ec30 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:45.589262 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57954c6-6ba2-44e7-82a5-d058cb54ec30 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:45.589757 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57954c6-6ba2-44e7-82a5-d058cb54ec30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:45.592148 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 48/90] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 17 02:58:45.603058 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57954c6-6ba2-44e7-82a5-d058cb54ec30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:58:45.608617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-d57954c6-6ba2-44e7-82a5-d058cb54ec30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:58:45.608617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:45.608617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:45.608617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:45.608617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:45.608617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:45.608617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:45.608617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:45.608617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:45.608617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:45.608617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:45.608617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:45.608617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:58:45.608617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:58:45.608617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:58:45.608617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:58:45.609446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:58:45.609446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:58:45.609446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:58:45.609446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:58:45.609446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:45.609446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:45.609446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:45.609446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:45.609446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:45.609446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:45.609446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:45.609446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:45.609446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:45.609446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:45.609446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:45.609446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:45.610207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:45.610207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:45.610207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:45.610207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:45.610207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:58:45.610207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:58:45.610207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:58:45.610207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:58:45.610207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:58:45.610207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:58:45.610207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:58:45.610207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:58:45.612404 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 43/91] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 02:58:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:58:45.618938 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e490eed-519c-415a-bfc2-402563449f66 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:45.749524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e490eed-519c-415a-bfc2-402563449f66 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:45.785522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa89e8-44fb-4088-85ba-eed57a50ede5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:45.786483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa89e8-44fb-4088-85ba-eed57a50ede5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:45.787013 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa89e8-44fb-4088-85ba-eed57a50ede5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:45.787497 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa89e8-44fb-4088-85ba-eed57a50ede5 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:45.788371 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa89e8-44fb-4088-85ba-eed57a50ede5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:45.788868 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa89e8-44fb-4088-85ba-eed57a50ede5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:45.790502 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa89e8-44fb-4088-85ba-eed57a50ede5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:45.799204 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 49/92] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:58:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:45.808196 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57954c6-6ba2-44e7-82a5-d058cb54ec30 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:45.924029 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57954c6-6ba2-44e7-82a5-d058cb54ec30 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:45.927291 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:45.946101 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f7506-0148-4471-b980-8b5b980703f0 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:45.947051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f7506-0148-4471-b980-8b5b980703f0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:45.947545 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f7506-0148-4471-b980-8b5b980703f0 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:45.949305 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 50/93] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:45.954020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad7230-5dd3-47ee-8afd-3a01c66b50cf None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:45.954517 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad7230-5dd3-47ee-8afd-3a01c66b50cf None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:45.955029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad7230-5dd3-47ee-8afd-3a01c66b50cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:45.967025 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:45.980209 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-cfad7230-5dd3-47ee-8afd-3a01c66b50cf None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:45.981787 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cfad7230-5dd3-47ee-8afd-3a01c66b50cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:46.008525 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b703c47-168f-4c4f-b078-49c2a9aef2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:46.009404 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b703c47-168f-4c4f-b078-49c2a9aef2ae None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:46.009835 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b703c47-168f-4c4f-b078-49c2a9aef2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:46.010258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b703c47-168f-4c4f-b078-49c2a9aef2ae None admin] PATH_INFO: `/v3/users/f84d693b9d0643c69861a5541952b43b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:46.015542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b703c47-168f-4c4f-b078-49c2a9aef2ae None admin] RBAC: Authorizing `identity:get_user(user_id=f84d693b9d0643c69861a5541952b43b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:46.017158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b703c47-168f-4c4f-b078-49c2a9aef2ae None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:46.018941 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 44/94] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:58:45 2022] GET /identity/v3/users/f84d693b9d0643c69861a5541952b43b => generated 247 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:46.021244 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990aff06-ec1e-4d3d-a6fa-62b36f31d49c None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:46.021688 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990aff06-ec1e-4d3d-a6fa-62b36f31d49c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:46.022140 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990aff06-ec1e-4d3d-a6fa-62b36f31d49c None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:46.027782 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 45/95] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:46.028743 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990aff06-ec1e-4d3d-a6fa-62b36f31d49c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:46.061755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 51/96] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:58:46.063436 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30907d-a31d-476e-aeb7-300287e02b97 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:46.063803 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30907d-a31d-476e-aeb7-300287e02b97 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:46.064155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30907d-a31d-476e-aeb7-300287e02b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:46.085353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-df30907d-a31d-476e-aeb7-300287e02b97 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:46.086719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-df30907d-a31d-476e-aeb7-300287e02b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:46.168207 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 52/97] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:46.171110 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30907d-a31d-476e-aeb7-300287e02b97 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:46.215687 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990aff06-ec1e-4d3d-a6fa-62b36f31d49c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:46.246538 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:46.277848 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077d625-7fee-4239-9774-9670b8bc0f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:46.278983 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077d625-7fee-4239-9774-9670b8bc0f5c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:46.279210 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077d625-7fee-4239-9774-9670b8bc0f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:46.283414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077d625-7fee-4239-9774-9670b8bc0f5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:46.330676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30907d-a31d-476e-aeb7-300287e02b97 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:46.366444 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:46.410654 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47514b5f-8068-48df-960a-2612c23a2a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:46.411514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47514b5f-8068-48df-960a-2612c23a2a94 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:46.411806 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47514b5f-8068-48df-960a-2612c23a2a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:46.412085 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47514b5f-8068-48df-960a-2612c23a2a94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:46.426450 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077d625-7fee-4239-9774-9670b8bc0f5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:46.428082 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077d625-7fee-4239-9774-9670b8bc0f5c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:46.430798 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47514b5f-8068-48df-960a-2612c23a2a94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:58:46.436737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-47514b5f-8068-48df-960a-2612c23a2a94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:58:46.436737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:46.436737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:46.436737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:46.436737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:46.436737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:46.436737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:46.436737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:46.436737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:46.436737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:46.436737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:46.436737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:46.436737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:58:46.436737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:58:46.436737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:58:46.436737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:58:46.437585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:58:46.437585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:58:46.437585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:58:46.437585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:58:46.437585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:46.437585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:46.437585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:46.437585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:46.437585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:46.437585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:46.437585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:46.437585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:46.437585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:46.437585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:46.437585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:46.437585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:46.438376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:46.438376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:46.438376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:46.438376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:46.438376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:58:46.438376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:58:46.438376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:58:46.438376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:58:46.438376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:58:46.438376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:58:46.438376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:58:46.438376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:58:46.443393 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 53/98] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:58:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:58:46.445740 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47514b5f-8068-48df-960a-2612c23a2a94 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:46.592292 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47514b5f-8068-48df-960a-2612c23a2a94 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:46.618181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:46.635011 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a183ff-b983-4aba-a489-c6b121879fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:46.635993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a183ff-b983-4aba-a489-c6b121879fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:46.636426 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a183ff-b983-4aba-a489-c6b121879fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:46.636861 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a183ff-b983-4aba-a489-c6b121879fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:46.637710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a183ff-b983-4aba-a489-c6b121879fc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:46.638169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a183ff-b983-4aba-a489-c6b121879fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:46.640599 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a183ff-b983-4aba-a489-c6b121879fc2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:46.654456 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 54/99] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:58:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:46.655614 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a183ff-b983-4aba-a489-c6b121879fc2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:46.676507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 46/100] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:58:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 02:58:46.677673 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077d625-7fee-4239-9774-9670b8bc0f5c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:46.776760 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:46.791555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a183ff-b983-4aba-a489-c6b121879fc2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:46.808886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077d625-7fee-4239-9774-9670b8bc0f5c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:46.828393 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdc5a5-29be-4396-9674-47f656f4a2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:46.829063 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdc5a5-29be-4396-9674-47f656f4a2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:46.829433 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdc5a5-29be-4396-9674-47f656f4a2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:46.829710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdc5a5-29be-4396-9674-47f656f4a2e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:46.841543 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fdc5a5-29be-4396-9674-47f656f4a2e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:58:46.843620 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9775e96-490d-4c60-ac96-f4398a18ef85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:46.844299 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9775e96-490d-4c60-ac96-f4398a18ef85 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:46.844588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9775e96-490d-4c60-ac96-f4398a18ef85 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:46.844869 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9775e96-490d-4c60-ac96-f4398a18ef85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:46.847108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-42fdc5a5-29be-4396-9674-47f656f4a2e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:58:46.847108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:46.847108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:46.847108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:46.847108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:46.847108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:46.847108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:46.847108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:46.847108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:46.847108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:46.847108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:46.847108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:46.847108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:58:46.847108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:58:46.847108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:58:46.847108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:58:46.848103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:58:46.848103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:58:46.848103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:58:46.848103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:58:46.848103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:46.848103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:46.848103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:46.848103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:46.848103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:46.848103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:46.848103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:46.848103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:46.848103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:46.848103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:46.848103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:46.848103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:46.849018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:46.849018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:46.849018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:46.849018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:46.849018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:58:46.849018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:58:46.849018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:58:46.849018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:58:46.849018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:58:46.849018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:58:46.849018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:58:46.849018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:58:46.849018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 55/101] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Oct 17 02:58:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:58:46.849642 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdc5a5-29be-4396-9674-47f656f4a2e8 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:46.858679 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9775e96-490d-4c60-ac96-f4398a18ef85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:58:46.864065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-e9775e96-490d-4c60-ac96-f4398a18ef85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:58:46.864065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:46.864065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:46.864065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:46.864065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:46.864065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:46.864065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:46.864065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:46.864065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:46.864065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:46.864065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:46.864065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:46.864065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:58:46.864065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:58:46.864065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:58:46.864065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:58:46.865023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:58:46.865023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:58:46.865023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:58:46.865023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:58:46.865023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:46.865023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:46.865023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:46.865023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:46.865023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:46.865023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:46.865023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:46.865023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:46.865023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:46.865023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:46.865023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:46.865023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:46.865795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:46.865795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:46.865795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:46.865795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:46.865795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:58:46.865795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:58:46.865795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:58:46.865795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:58:46.865795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:58:46.865795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:58:46.865795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:58:46.865795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:58:46.865795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 47/102] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:58:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:58:46.867395 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9775e96-490d-4c60-ac96-f4398a18ef85 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:46.896768 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:46.948523 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdc5a5-29be-4396-9674-47f656f4a2e8 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:46.966276 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9775e96-490d-4c60-ac96-f4398a18ef85 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:46.980710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a3ce9-1e99-4a75-900c-d58b1298c2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:46.981338 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a3ce9-1e99-4a75-900c-d58b1298c2cf None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:46.981622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a3ce9-1e99-4a75-900c-d58b1298c2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:46.981898 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a3ce9-1e99-4a75-900c-d58b1298c2cf None admin] PATH_INFO: `/v3/users/f5d5caad74e9468ca5bba4cd444b7404` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:46.983048 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a3ce9-1e99-4a75-900c-d58b1298c2cf None admin] RBAC: Authorizing `identity:get_user(user_id=f5d5caad74e9468ca5bba4cd444b7404)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:46.984485 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a3ce9-1e99-4a75-900c-d58b1298c2cf None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:46.986189 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 56/103] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:58:46 2022] GET /identity/v3/users/f5d5caad74e9468ca5bba4cd444b7404 => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:46.987170 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14470fbc-91c7-4d98-bdfc-6337549e4a5d None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:46.987453 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14470fbc-91c7-4d98-bdfc-6337549e4a5d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:46.987728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14470fbc-91c7-4d98-bdfc-6337549e4a5d None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:46.989685 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 57/104] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:46.990697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940419b-54b5-4e01-8746-82e001f869e6 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:46.991005 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940419b-54b5-4e01-8746-82e001f869e6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:46.991280 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940419b-54b5-4e01-8746-82e001f869e6 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:46.992642 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 58/105] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:46.995627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940419b-54b5-4e01-8746-82e001f869e6 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:47.000307 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:47.006553 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf83b26-abf7-410a-8f4d-b7e7419d3627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:47.007222 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf83b26-abf7-410a-8f4d-b7e7419d3627 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:47.007509 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf83b26-abf7-410a-8f4d-b7e7419d3627 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:47.007786 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf83b26-abf7-410a-8f4d-b7e7419d3627 None admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:47.008923 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf83b26-abf7-410a-8f4d-b7e7419d3627 None admin] RBAC: Authorizing `identity:get_project(project_id=f64aa9442bc9483987f3c997119762af)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:47.010389 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf83b26-abf7-410a-8f4d-b7e7419d3627 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:47.012063 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 48/106] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:58:46 2022] GET /identity/v3/projects/f64aa9442bc9483987f3c997119762af => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:47.012063 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf83b26-abf7-410a-8f4d-b7e7419d3627 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:47.103609 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940419b-54b5-4e01-8746-82e001f869e6 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:47.109304 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf83b26-abf7-410a-8f4d-b7e7419d3627 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:47.116754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:47.137230 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3216cb-0700-48a6-8f31-776f9cc66769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:47.137842 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3216cb-0700-48a6-8f31-776f9cc66769 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:47.138120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3216cb-0700-48a6-8f31-776f9cc66769 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:47.138409 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3216cb-0700-48a6-8f31-776f9cc66769 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:47.139099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3216cb-0700-48a6-8f31-776f9cc66769 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:47.139414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3216cb-0700-48a6-8f31-776f9cc66769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:47.140788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3216cb-0700-48a6-8f31-776f9cc66769 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:47.142813 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b476d-7994-4a58-a327-4503cddd2908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:47.143439 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b476d-7994-4a58-a327-4503cddd2908 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:47.143720 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b476d-7994-4a58-a327-4503cddd2908 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:47.143993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b476d-7994-4a58-a327-4503cddd2908 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:47.144655 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b476d-7994-4a58-a327-4503cddd2908 None admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:47.145893 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b476d-7994-4a58-a327-4503cddd2908 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:47.146965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b476d-7994-4a58-a327-4503cddd2908 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:47.149624 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 59/107] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:58:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:47.150301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3216cb-0700-48a6-8f31-776f9cc66769 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:47.157357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 49/108] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:58:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:47.160295 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eec656-5073-480f-bca0-dcefc712ce80 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:47.164478 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eec656-5073-480f-bca0-dcefc712ce80 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:47.164780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eec656-5073-480f-bca0-dcefc712ce80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:47.184352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-45eec656-5073-480f-bca0-dcefc712ce80 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:47.185593 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-45eec656-5073-480f-bca0-dcefc712ce80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:47.245235 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:47.249602 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3216cb-0700-48a6-8f31-776f9cc66769 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:47.256515 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 50/109] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:47.257840 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339806a-bd3a-4126-b251-798cd12f2d02 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:47.258138 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339806a-bd3a-4126-b251-798cd12f2d02 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:47.258419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339806a-bd3a-4126-b251-798cd12f2d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:47.276276 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9339806a-bd3a-4126-b251-798cd12f2d02 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:47.277500 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9339806a-bd3a-4126-b251-798cd12f2d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:47.296267 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb635990-fc32-43db-be8c-04c66e8bc080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:47.297134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb635990-fc32-43db-be8c-04c66e8bc080 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:47.297422 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb635990-fc32-43db-be8c-04c66e8bc080 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:47.297698 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb635990-fc32-43db-be8c-04c66e8bc080 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:47.346499 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:47.349172 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 51/110] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:47.350397 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339806a-bd3a-4126-b251-798cd12f2d02 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:47.376733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb635990-fc32-43db-be8c-04c66e8bc080 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:47.378840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb635990-fc32-43db-be8c-04c66e8bc080 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:47.454327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339806a-bd3a-4126-b251-798cd12f2d02 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:47.478025 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:47.490642 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020fd9c-6d4a-4620-924b-3a4a16092553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:47.491294 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020fd9c-6d4a-4620-924b-3a4a16092553 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:47.491583 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020fd9c-6d4a-4620-924b-3a4a16092553 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:47.491862 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020fd9c-6d4a-4620-924b-3a4a16092553 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:47.492598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020fd9c-6d4a-4620-924b-3a4a16092553 None admin] RBAC: Adding query filter params (role.id=ea0fd0c10f2f4e529270edbe3525d4c6, scope.project.id=f64aa9442bc9483987f3c997119762af, user.id=f84d693b9d0643c69861a5541952b43b) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:47.492912 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020fd9c-6d4a-4620-924b-3a4a16092553 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:47.493970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 60/111] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:58:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 02:58:47.498551 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7020fd9c-6d4a-4620-924b-3a4a16092553 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:47.499784 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb635990-fc32-43db-be8c-04c66e8bc080 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:47.501728 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 52/112] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 02:58:47 2022] GET /identity/v3/role_assignments?user.id=f84d693b9d0643c69861a5541952b43b&scope.project.id=f64aa9442bc9483987f3c997119762af&role.id=ea0fd0c10f2f4e529270edbe3525d4c6 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:47.502584 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020fd9c-6d4a-4620-924b-3a4a16092553 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:47.603434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:47.615833 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020fd9c-6d4a-4620-924b-3a4a16092553 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:47.631043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb635990-fc32-43db-be8c-04c66e8bc080 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:47.661028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846588b3-dfed-44d3-802e-cde70939d34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:47.661683 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846588b3-dfed-44d3-802e-cde70939d34b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:47.661968 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846588b3-dfed-44d3-802e-cde70939d34b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:47.662245 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846588b3-dfed-44d3-802e-cde70939d34b None admin] PATH_INFO: `/v3/users/f5d5caad74e9468ca5bba4cd444b7404` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:47.663491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846588b3-dfed-44d3-802e-cde70939d34b None admin] RBAC: Authorizing `identity:get_user(user_id=f5d5caad74e9468ca5bba4cd444b7404)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:47.665009 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846588b3-dfed-44d3-802e-cde70939d34b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:47.666710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 53/113] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:58:47 2022] GET /identity/v3/users/f5d5caad74e9468ca5bba4cd444b7404 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:47.667526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846588b3-dfed-44d3-802e-cde70939d34b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:47.677218 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9b060-f99e-4cfd-bca9-c7aa3b6a05a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:47.678099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9b060-f99e-4cfd-bca9-c7aa3b6a05a0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:47.678382 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9b060-f99e-4cfd-bca9-c7aa3b6a05a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:47.678659 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9b060-f99e-4cfd-bca9-c7aa3b6a05a0 None admin] PATH_INFO: `/v3/users/f5d5caad74e9468ca5bba4cd444b7404` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:47.679904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9b060-f99e-4cfd-bca9-c7aa3b6a05a0 None admin] RBAC: Authorizing `identity:get_user(user_id=f5d5caad74e9468ca5bba4cd444b7404)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:47.681699 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9b060-f99e-4cfd-bca9-c7aa3b6a05a0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:47.683584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 61/114] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:58:47 2022] GET /identity/v3/users/f5d5caad74e9468ca5bba4cd444b7404 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:47.684307 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9b060-f99e-4cfd-bca9-c7aa3b6a05a0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:47.739750 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:47.789940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846588b3-dfed-44d3-802e-cde70939d34b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:47.796196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9b060-f99e-4cfd-bca9-c7aa3b6a05a0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:47.824730 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6655f47-b6e4-43ae-9646-a86011c6dfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:47.825363 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6655f47-b6e4-43ae-9646-a86011c6dfec None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:47.825651 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6655f47-b6e4-43ae-9646-a86011c6dfec None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:47.825928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6655f47-b6e4-43ae-9646-a86011c6dfec None admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:47.827090 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6655f47-b6e4-43ae-9646-a86011c6dfec None admin] RBAC: Authorizing `identity:get_project(project_id=f64aa9442bc9483987f3c997119762af)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:47.828540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6655f47-b6e4-43ae-9646-a86011c6dfec None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:47.830078 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f682ac1-fa26-4124-97ad-d699cccf45c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:47.830175 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 54/115] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:58:47 2022] GET /identity/v3/projects/f64aa9442bc9483987f3c997119762af => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:47.830718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f682ac1-fa26-4124-97ad-d699cccf45c0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:47.830946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6655f47-b6e4-43ae-9646-a86011c6dfec None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:47.831037 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f682ac1-fa26-4124-97ad-d699cccf45c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:47.831315 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f682ac1-fa26-4124-97ad-d699cccf45c0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:47.844640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f682ac1-fa26-4124-97ad-d699cccf45c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:58:47.850057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-7f682ac1-fa26-4124-97ad-d699cccf45c0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 02:58:47.850057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:47.850057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:47.850057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:47.850057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:47.850057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:47.850057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:47.850057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:47.850057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:47.850057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:47.850057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:47.850057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:47.850057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:58:47.850057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:58:47.850057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:58:47.850057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:58:47.851126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:58:47.851126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:58:47.851126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:58:47.851126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:58:47.851126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:47.851126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:47.851126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:47.851126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:47.851126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:47.851126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:47.851126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:47.851126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:47.851126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:47.851126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:47.851126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:47.851126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:47.855700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:47.855700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:47.855700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:47.855700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:47.855700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:58:47.855700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:58:47.855700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:58:47.855700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:58:47.855700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:58:47.855700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:58:47.855700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 02:58:47.855700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:58:47.855700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 62/116] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:58:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:58:47.855700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f682ac1-fa26-4124-97ad-d699cccf45c0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:47.856395 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:47.936835 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6655f47-b6e4-43ae-9646-a86011c6dfec None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:47.951355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f682ac1-fa26-4124-97ad-d699cccf45c0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:47.958435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:47.971043 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da264383-9809-419c-9f81-a09e882532f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:47.971752 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da264383-9809-419c-9f81-a09e882532f6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:47.972041 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da264383-9809-419c-9f81-a09e882532f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:47.972317 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da264383-9809-419c-9f81-a09e882532f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:47.974104 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da264383-9809-419c-9f81-a09e882532f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:47.975610 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da264383-9809-419c-9f81-a09e882532f6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:47.977354 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 55/117] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:58:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:47.977914 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da264383-9809-419c-9f81-a09e882532f6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:47.985773 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b16a6-8def-4551-8b61-1e8f93428406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:47.986392 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b16a6-8def-4551-8b61-1e8f93428406 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:47.986679 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b16a6-8def-4551-8b61-1e8f93428406 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:47.986986 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b16a6-8def-4551-8b61-1e8f93428406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:48.055459 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b16a6-8def-4551-8b61-1e8f93428406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:48.056946 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64b16a6-8def-4551-8b61-1e8f93428406 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:48.059569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:48.078705 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da264383-9809-419c-9f81-a09e882532f6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:48.111392 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babc8bf-265d-4597-ac76-3bf7f0c24d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:48.111992 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babc8bf-265d-4597-ac76-3bf7f0c24d9c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:48.112276 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babc8bf-265d-4597-ac76-3bf7f0c24d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:48.112552 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3babc8bf-265d-4597-ac76-3bf7f0c24d9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:48.157043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 63/118] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:58:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 02:58:48.157884 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b16a6-8def-4551-8b61-1e8f93428406 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:48.177820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:48.188589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babc8bf-265d-4597-ac76-3bf7f0c24d9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:48.189994 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3babc8bf-265d-4597-ac76-3bf7f0c24d9c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:48.261639 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b16a6-8def-4551-8b61-1e8f93428406 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:48.263923 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:48.296590 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba37b1-03e8-46ab-8788-85f4992cd7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:48.297201 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba37b1-03e8-46ab-8788-85f4992cd7ea None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:48.297370 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 56/119] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:58:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 02:58:48.297464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba37b1-03e8-46ab-8788-85f4992cd7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:48.297741 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba37b1-03e8-46ab-8788-85f4992cd7ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:48.298210 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babc8bf-265d-4597-ac76-3bf7f0c24d9c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:48.298537 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba37b1-03e8-46ab-8788-85f4992cd7ea None admin] RBAC: Adding query filter params (role.id=11de9783bae3450ea53955de9ebcc9d1, scope.project.id=f64aa9442bc9483987f3c997119762af, user.id=f5d5caad74e9468ca5bba4cd444b7404) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:48.298849 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba37b1-03e8-46ab-8788-85f4992cd7ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:48.300325 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba37b1-03e8-46ab-8788-85f4992cd7ea None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:48.308583 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 64/120] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 02:58:48 2022] GET /identity/v3/role_assignments?user.id=f5d5caad74e9468ca5bba4cd444b7404&scope.project.id=f64aa9442bc9483987f3c997119762af&role.id=11de9783bae3450ea53955de9ebcc9d1 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:48.309409 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba37b1-03e8-46ab-8788-85f4992cd7ea None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:48.400162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:48.412850 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3babc8bf-265d-4597-ac76-3bf7f0c24d9c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:48.428804 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba37b1-03e8-46ab-8788-85f4992cd7ea None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:48.448181 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313beb9-99dd-4476-b184-329e89499b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:48.448734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313beb9-99dd-4476-b184-329e89499b0c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:48.448999 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313beb9-99dd-4476-b184-329e89499b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:48.449268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313beb9-99dd-4476-b184-329e89499b0c None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:48.449941 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b313beb9-99dd-4476-b184-329e89499b0c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:48.450249 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b313beb9-99dd-4476-b184-329e89499b0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:48.451710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b313beb9-99dd-4476-b184-329e89499b0c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:48.461369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 57/121] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:58:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:48.462146 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313beb9-99dd-4476-b184-329e89499b0c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:48.470970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43822e-d712-4fa1-97d7-5686a28080af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:48.471617 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43822e-d712-4fa1-97d7-5686a28080af None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:48.471907 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43822e-d712-4fa1-97d7-5686a28080af None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:48.472188 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43822e-d712-4fa1-97d7-5686a28080af None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:48.472884 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43822e-d712-4fa1-97d7-5686a28080af None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:48.474915 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43822e-d712-4fa1-97d7-5686a28080af None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:48.506377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 65/122] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Oct 17 02:58:48 2022] POST /identity/v3/projects => generated 302 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:58:48.507277 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43822e-d712-4fa1-97d7-5686a28080af None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:48.534619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:48.601656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313beb9-99dd-4476-b184-329e89499b0c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:48.639667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43822e-d712-4fa1-97d7-5686a28080af None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:48.646507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e369d-caa6-459e-85bf-0ed5e046513c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:48.648108 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e369d-caa6-459e-85bf-0ed5e046513c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:48.648587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e369d-caa6-459e-85bf-0ed5e046513c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:48.649047 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e369d-caa6-459e-85bf-0ed5e046513c None admin] PATH_INFO: `/v3/projects/b5cfbc27829541e280a643a64f865804` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:48.650852 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e369d-caa6-459e-85bf-0ed5e046513c None admin] RBAC: Authorizing `identity:get_project(project_id=b5cfbc27829541e280a643a64f865804)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:48.653305 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e369d-caa6-459e-85bf-0ed5e046513c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:48.656239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 58/123] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:58:48 2022] GET /identity/v3/projects/b5cfbc27829541e280a643a64f865804 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:48.657403 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e369d-caa6-459e-85bf-0ed5e046513c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:48.676956 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af230ae4-01d6-43d7-92dc-4bf390e901fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:48.677610 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af230ae4-01d6-43d7-92dc-4bf390e901fd None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:48.678216 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af230ae4-01d6-43d7-92dc-4bf390e901fd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:48.678497 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af230ae4-01d6-43d7-92dc-4bf390e901fd None admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:48.679871 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af230ae4-01d6-43d7-92dc-4bf390e901fd None admin] RBAC: Authorizing `identity:get_project(project_id=f64aa9442bc9483987f3c997119762af)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:48.681347 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af230ae4-01d6-43d7-92dc-4bf390e901fd None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:48.683866 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 66/124] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:58:48 2022] GET /identity/v3/projects/f64aa9442bc9483987f3c997119762af => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:48.683998 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af230ae4-01d6-43d7-92dc-4bf390e901fd None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:48.692930 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:48.778071 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:48.782135 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e369d-caa6-459e-85bf-0ed5e046513c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:48.793222 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af230ae4-01d6-43d7-92dc-4bf390e901fd None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:48.818832 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae839e-cce4-4037-aba3-9611bcae2971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:48.819504 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae839e-cce4-4037-aba3-9611bcae2971 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:48.819794 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae839e-cce4-4037-aba3-9611bcae2971 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:48.820077 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae839e-cce4-4037-aba3-9611bcae2971 None admin] PATH_INFO: `/v3/users/9573316208b6446c880e2505aacfdd0e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:48.830360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe330e-ff73-4854-ad37-8eee26a5a80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:48.831330 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe330e-ff73-4854-ad37-8eee26a5a80a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:48.831851 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe330e-ff73-4854-ad37-8eee26a5a80a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:48.832898 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe330e-ff73-4854-ad37-8eee26a5a80a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:48.834056 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe330e-ff73-4854-ad37-8eee26a5a80a None admin] RBAC: Adding query filter params (role.id=11de9783bae3450ea53955de9ebcc9d1, scope.project.id=b5cfbc27829541e280a643a64f865804, user.id=f5d5caad74e9468ca5bba4cd444b7404) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:48.834264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae839e-cce4-4037-aba3-9611bcae2971 None admin] RBAC: Authorizing `identity:get_user(user_id=9573316208b6446c880e2505aacfdd0e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:48.834727 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe330e-ff73-4854-ad37-8eee26a5a80a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:48.835784 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae839e-cce4-4037-aba3-9611bcae2971 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:48.836547 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe330e-ff73-4854-ad37-8eee26a5a80a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:48.837437 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 59/125] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:58:48 2022] GET /identity/v3/users/9573316208b6446c880e2505aacfdd0e => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:48.839866 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae839e-cce4-4037-aba3-9611bcae2971 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:48.844975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 67/126] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 02:58:48 2022] GET /identity/v3/role_assignments?user.id=f5d5caad74e9468ca5bba4cd444b7404&scope.project.id=b5cfbc27829541e280a643a64f865804&role.id=11de9783bae3450ea53955de9ebcc9d1 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:48.845996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe330e-ff73-4854-ad37-8eee26a5a80a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:48.941086 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:48.958347 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe330e-ff73-4854-ad37-8eee26a5a80a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:48.968623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae839e-cce4-4037-aba3-9611bcae2971 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:48.996706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa09def-cd25-4865-97d6-6b87cc5f1922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:48.997482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa09def-cd25-4865-97d6-6b87cc5f1922 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:48.997798 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa09def-cd25-4865-97d6-6b87cc5f1922 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:48.998084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa09def-cd25-4865-97d6-6b87cc5f1922 None admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:48.999354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa09def-cd25-4865-97d6-6b87cc5f1922 None admin] RBAC: Authorizing `identity:get_project(project_id=f64aa9442bc9483987f3c997119762af)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:49.000860 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa09def-cd25-4865-97d6-6b87cc5f1922 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:49.002763 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 68/127] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:58:48 2022] GET /identity/v3/projects/f64aa9442bc9483987f3c997119762af => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:49.009540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa09def-cd25-4865-97d6-6b87cc5f1922 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:49.016314 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5d5f8-150f-49c2-8b09-28079de3571c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:49.016938 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5d5f8-150f-49c2-8b09-28079de3571c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:49.017223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5d5f8-150f-49c2-8b09-28079de3571c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:49.017502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5d5f8-150f-49c2-8b09-28079de3571c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:49.018250 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5d5f8-150f-49c2-8b09-28079de3571c None admin] RBAC: Adding query filter params (role.id=baf207ce98e448f1a2e10c3310a0e33b, scope.project.id=f64aa9442bc9483987f3c997119762af, user.id=f5d5caad74e9468ca5bba4cd444b7404) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:49.018560 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5d5f8-150f-49c2-8b09-28079de3571c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:49.020684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5d5f8-150f-49c2-8b09-28079de3571c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:49.028840 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 60/128] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 02:58:48 2022] GET /identity/v3/role_assignments?user.id=f5d5caad74e9468ca5bba4cd444b7404&scope.project.id=f64aa9442bc9483987f3c997119762af&role.id=baf207ce98e448f1a2e10c3310a0e33b => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:49.036084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:49.140620 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa09def-cd25-4865-97d6-6b87cc5f1922 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:49.178447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebdbad7-0067-4ce2-bb2e-7e5e2302f500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:49.179167 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdbad7-0067-4ce2-bb2e-7e5e2302f500 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:49.179452 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdbad7-0067-4ce2-bb2e-7e5e2302f500 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:49.179727 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdbad7-0067-4ce2-bb2e-7e5e2302f500 None admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af/users/9573316208b6446c880e2505aacfdd0e/roles/1dc8aea1b26145879f699bdeb2f077da` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:49.181699 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebdbad7-0067-4ce2-bb2e-7e5e2302f500 None admin] RBAC: Authorizing `identity:create_grant(project_id=f64aa9442bc9483987f3c997119762af, user_id=9573316208b6446c880e2505aacfdd0e, role_id=1dc8aea1b26145879f699bdeb2f077da)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:49.183209 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebdbad7-0067-4ce2-bb2e-7e5e2302f500 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:49.205739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 69/129] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 02:58:49 2022] PUT /identity/v3/projects/f64aa9442bc9483987f3c997119762af/users/9573316208b6446c880e2505aacfdd0e/roles/1dc8aea1b26145879f699bdeb2f077da => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 02:58:50.711091 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fda608-0fca-48fb-a88c-5536e7e0704b None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:50.711607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fda608-0fca-48fb-a88c-5536e7e0704b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:50.711688 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fda608-0fca-48fb-a88c-5536e7e0704b None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:50.713027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 61/130] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:50.718320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a557941-4ffe-4ff7-91b2-580638a1fb23 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:50.718707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a557941-4ffe-4ff7-91b2-580638a1fb23 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:50.718909 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a557941-4ffe-4ff7-91b2-580638a1fb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:50.737244 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8a557941-4ffe-4ff7-91b2-580638a1fb23 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:50.738427 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8a557941-4ffe-4ff7-91b2-580638a1fb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:50.802796 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 70/131] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:51.070741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fda608-0fca-48fb-a88c-5536e7e0704b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:51.170790 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fda608-0fca-48fb-a88c-5536e7e0704b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:51.202799 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451c0e3-f4cf-4320-8c5c-aa222788ee02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:51.203458 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451c0e3-f4cf-4320-8c5c-aa222788ee02 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:51.203741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451c0e3-f4cf-4320-8c5c-aa222788ee02 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:51.204014 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451c0e3-f4cf-4320-8c5c-aa222788ee02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:51.205125 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451c0e3-f4cf-4320-8c5c-aa222788ee02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:51.206576 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451c0e3-f4cf-4320-8c5c-aa222788ee02 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:51.208802 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 62/132] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:58:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:51.215063 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a557941-4ffe-4ff7-91b2-580638a1fb23 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:51.226965 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:51.321044 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a557941-4ffe-4ff7-91b2-580638a1fb23 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:51.354388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7e4b3b-08c0-42d3-a2fc-0605475d8398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:51.355035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e4b3b-08c0-42d3-a2fc-0605475d8398 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:51.355318 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e4b3b-08c0-42d3-a2fc-0605475d8398 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:51.355588 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7e4b3b-08c0-42d3-a2fc-0605475d8398 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:51.356245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e4b3b-08c0-42d3-a2fc-0605475d8398 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:51.357677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7e4b3b-08c0-42d3-a2fc-0605475d8398 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:51.385976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 71/133] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 02:58:51 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:58:51.403372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264c172-943b-4654-b3c8-844581d8d00e None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:51.403761 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264c172-943b-4654-b3c8-844581d8d00e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:51.404133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264c172-943b-4654-b3c8-844581d8d00e None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:51.405643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 63/134] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:51.412918 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3a455-1ad7-43ca-b8d5-beaa475d9463 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:51.413238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3a455-1ad7-43ca-b8d5-beaa475d9463 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:51.413523 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3a455-1ad7-43ca-b8d5-beaa475d9463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:51.432728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f4e3a455-1ad7-43ca-b8d5-beaa475d9463 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:51.433950 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f4e3a455-1ad7-43ca-b8d5-beaa475d9463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:51.497533 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:51.510924 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 72/135] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:51.789015 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264c172-943b-4654-b3c8-844581d8d00e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:51.803184 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:51.904338 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264c172-943b-4654-b3c8-844581d8d00e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:51.938887 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b033b0-bd24-415e-80f4-ea34d5ed6ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:51.939532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b033b0-bd24-415e-80f4-ea34d5ed6ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:51.939813 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b033b0-bd24-415e-80f4-ea34d5ed6ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:51.940139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b033b0-bd24-415e-80f4-ea34d5ed6ef9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:51.951740 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b033b0-bd24-415e-80f4-ea34d5ed6ef9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:58:51.957150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-74b033b0-bd24-415e-80f4-ea34d5ed6ef9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 02:58:51.957150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:51.957150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:51.957150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:51.957150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:51.957150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:51.957150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:51.957150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:51.957150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:51.957150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:51.957150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:51.957150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:51.957150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:58:51.957150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:58:51.957150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:58:51.957150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:58:51.958114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:58:51.958114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:58:51.958114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:58:51.958114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:58:51.958114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:51.958114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:51.958114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:51.958114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:51.958114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:51.958114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:51.958114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:51.958114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:51.958114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:51.958114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:51.958114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:51.958114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:51.959052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:51.959052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:51.959052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:51.959052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:51.959052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:58:51.959052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:58:51.959052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:58:51.959052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:58:51.959052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:58:51.959052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:58:51.959052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 02:58:51.959052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:58:51.959052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 64/136] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:58:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:58:51.964353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3a455-1ad7-43ca-b8d5-beaa475d9463 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:52.065192 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3a455-1ad7-43ca-b8d5-beaa475d9463 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:52.105258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e0ae6-f40d-40a6-8703-4aac434130df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:52.105880 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e0ae6-f40d-40a6-8703-4aac434130df None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:52.106221 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e0ae6-f40d-40a6-8703-4aac434130df None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:52.106505 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e0ae6-f40d-40a6-8703-4aac434130df None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:52.107223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e0ae6-f40d-40a6-8703-4aac434130df None admin] RBAC: Adding query filter params (name=reader) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:52.107534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e0ae6-f40d-40a6-8703-4aac434130df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:52.108944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e0ae6-f40d-40a6-8703-4aac434130df None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:52.118538 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 73/137] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:58:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:52.124014 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b033b0-bd24-415e-80f4-ea34d5ed6ef9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:52.204976 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:52.241133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b033b0-bd24-415e-80f4-ea34d5ed6ef9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:52.281044 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ebfcf-d2a9-4348-97af-657bb9e23b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:52.281946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ebfcf-d2a9-4348-97af-657bb9e23b95 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:52.282428 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ebfcf-d2a9-4348-97af-657bb9e23b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:52.282976 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ebfcf-d2a9-4348-97af-657bb9e23b95 None admin] PATH_INFO: `/v3/users/9573316208b6446c880e2505aacfdd0e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:52.284336 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ebfcf-d2a9-4348-97af-657bb9e23b95 None admin] RBAC: Authorizing `identity:get_user(user_id=9573316208b6446c880e2505aacfdd0e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:52.286575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ebfcf-d2a9-4348-97af-657bb9e23b95 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:52.288721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 65/138] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:58:52 2022] GET /identity/v3/users/9573316208b6446c880e2505aacfdd0e => generated 289 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:52.294533 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e0ae6-f40d-40a6-8703-4aac434130df None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:52.354976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:52.403884 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e0ae6-f40d-40a6-8703-4aac434130df None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:52.438601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfd8cb-e14d-4220-8be6-e6181970d64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:52.439249 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfd8cb-e14d-4220-8be6-e6181970d64f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:52.439543 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfd8cb-e14d-4220-8be6-e6181970d64f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:52.439819 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfd8cb-e14d-4220-8be6-e6181970d64f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:52.513509 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfd8cb-e14d-4220-8be6-e6181970d64f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:52.515028 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfd8cb-e14d-4220-8be6-e6181970d64f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:52.573442 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:52.617559 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 74/139] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:58:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 02:58:52.623432 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ebfcf-d2a9-4348-97af-657bb9e23b95 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:52.729058 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ebfcf-d2a9-4348-97af-657bb9e23b95 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:52.763541 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8930d1-ebdc-451d-99d1-bc827d33d038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:52.764290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8930d1-ebdc-451d-99d1-bc827d33d038 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:52.764583 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8930d1-ebdc-451d-99d1-bc827d33d038 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:52.764866 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8930d1-ebdc-451d-99d1-bc827d33d038 None admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:52.766014 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8930d1-ebdc-451d-99d1-bc827d33d038 None admin] RBAC: Authorizing `identity:get_project(project_id=f64aa9442bc9483987f3c997119762af)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:52.767508 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8930d1-ebdc-451d-99d1-bc827d33d038 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:52.769164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 66/140] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:58:52 2022] GET /identity/v3/projects/f64aa9442bc9483987f3c997119762af => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:52.775992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfd8cb-e14d-4220-8be6-e6181970d64f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:52.877272 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfd8cb-e14d-4220-8be6-e6181970d64f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:52.912911 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9fca9-7338-429e-b31f-9669c857ff73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:52.913535 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9fca9-7338-429e-b31f-9669c857ff73 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:52.913892 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9fca9-7338-429e-b31f-9669c857ff73 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:52.914174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9fca9-7338-429e-b31f-9669c857ff73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:52.915003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9fca9-7338-429e-b31f-9669c857ff73 None admin] RBAC: Adding query filter params (role.id=1dc8aea1b26145879f699bdeb2f077da, scope.project.id=f64aa9442bc9483987f3c997119762af, user.id=9573316208b6446c880e2505aacfdd0e) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:52.915332 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9fca9-7338-429e-b31f-9669c857ff73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:52.916785 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9fca9-7338-429e-b31f-9669c857ff73 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:52.922644 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 75/141] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 02:58:52 2022] GET /identity/v3/role_assignments?user.id=9573316208b6446c880e2505aacfdd0e&scope.project.id=f64aa9442bc9483987f3c997119762af&role.id=1dc8aea1b26145879f699bdeb2f077da => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:53.670550 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8abb72-1cf9-4254-8bae-a80e149cae59 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:53.671126 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8abb72-1cf9-4254-8bae-a80e149cae59 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:53.671126 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8abb72-1cf9-4254-8bae-a80e149cae59 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:53.672758 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 67/142] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:53.678944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4963b-13b1-49e8-a878-4358753f624f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:53.679443 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4963b-13b1-49e8-a878-4358753f624f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:53.679510 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4963b-13b1-49e8-a878-4358753f624f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:53.699270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ffa4963b-13b1-49e8-a878-4358753f624f None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:53.700510 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ffa4963b-13b1-49e8-a878-4358753f624f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:53.719774 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:53.773475 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 76/143] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:53.797944 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f36b71-eba8-4470-9090-42548a1915d7 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:53.798238 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f36b71-eba8-4470-9090-42548a1915d7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:53.798695 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f36b71-eba8-4470-9090-42548a1915d7 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:53.800069 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 68/144] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:53.806008 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc5110-297b-4dd4-9715-2e6c9ef21b73 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:53.806299 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc5110-297b-4dd4-9715-2e6c9ef21b73 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:53.806574 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc5110-297b-4dd4-9715-2e6c9ef21b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:53.825685 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-24dc5110-297b-4dd4-9715-2e6c9ef21b73 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:53.826909 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-24dc5110-297b-4dd4-9715-2e6c9ef21b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:53.899224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 77/145] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:53.986003 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb2c16-94e0-456a-8236-edf27fc61db2 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:53.986304 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb2c16-94e0-456a-8236-edf27fc61db2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:53.986499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb2c16-94e0-456a-8236-edf27fc61db2 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:53.988010 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 69/146] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:53.992939 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef7ef0-8b7f-4ac2-a043-329eb912a60a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:53.993233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef7ef0-8b7f-4ac2-a043-329eb912a60a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:53.993506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef7ef0-8b7f-4ac2-a043-329eb912a60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:54.012161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-bdef7ef0-8b7f-4ac2-a043-329eb912a60a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:54.013330 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-bdef7ef0-8b7f-4ac2-a043-329eb912a60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:54.039181 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb2c16-94e0-456a-8236-edf27fc61db2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:54.048492 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:54.083280 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 78/147] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:54.142857 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb2c16-94e0-456a-8236-edf27fc61db2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:54.171288 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef7ef0-8b7f-4ac2-a043-329eb912a60a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:54.175885 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c70be-1570-4bc1-afd6-abb242d421a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:54.176765 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c70be-1570-4bc1-afd6-abb242d421a8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:54.177044 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c70be-1570-4bc1-afd6-abb242d421a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:54.177316 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c70be-1570-4bc1-afd6-abb242d421a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:54.185319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:54.189107 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c70be-1570-4bc1-afd6-abb242d421a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:58:54.195878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-fd6c70be-1570-4bc1-afd6-abb242d421a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:58:54.195878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:54.195878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:54.195878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:54.195878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:54.195878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:54.195878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:54.195878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:54.195878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:54.195878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:54.195878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:54.195878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:54.195878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:58:54.195878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:58:54.195878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:58:54.195878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:58:54.196721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:58:54.196721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:58:54.196721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:58:54.196721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:58:54.196721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:54.196721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:54.196721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:54.196721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:54.196721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:54.196721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:54.196721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:54.196721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:54.196721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:54.196721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:54.196721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:54.196721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:54.197886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:54.197886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:54.197886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:54.197886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:54.197886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:58:54.197886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:58:54.197886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:58:54.197886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:58:54.197886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:58:54.197886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:58:54.197886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:58:54.197886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:58:54.197886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 70/148] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 02:58:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:58:54.202947 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c70be-1570-4bc1-afd6-abb242d421a8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:54.277516 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:54.282715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef7ef0-8b7f-4ac2-a043-329eb912a60a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:54.317995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c70be-1570-4bc1-afd6-abb242d421a8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:54.322993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62c533-45fb-4a41-bc11-e1bf9abafba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:54.323638 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62c533-45fb-4a41-bc11-e1bf9abafba2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:54.323918 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62c533-45fb-4a41-bc11-e1bf9abafba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:54.324194 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62c533-45fb-4a41-bc11-e1bf9abafba2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:54.334968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62c533-45fb-4a41-bc11-e1bf9abafba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:58:54.339762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-eb62c533-45fb-4a41-bc11-e1bf9abafba2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:58:54.339762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:54.339762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:54.339762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:54.339762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:54.339762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:54.339762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:54.339762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:54.339762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:54.339762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:54.339762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:54.339762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:54.339762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:58:54.339762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:58:54.339762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:58:54.339762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:58:54.340625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:58:54.340625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:58:54.340625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:58:54.340625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:58:54.340625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:54.340625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:54.340625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:54.340625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:54.340625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:54.340625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:54.340625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:54.340625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:54.340625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:54.340625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:54.340625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:54.340625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:54.341407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:54.341407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:54.341407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:54.341407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:54.341407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:58:54.341407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:58:54.341407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:58:54.341407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:58:54.341407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:58:54.341407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:58:54.341407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:58:54.341407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:58:54.341407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 79/149] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Oct 17 02:58:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:58:54.346486 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62c533-45fb-4a41-bc11-e1bf9abafba2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:54.357227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb49f2-d9ea-4705-96e3-b4c877182eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:54.358377 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb49f2-d9ea-4705-96e3-b4c877182eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:54.359252 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb49f2-d9ea-4705-96e3-b4c877182eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:54.359694 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb49f2-d9ea-4705-96e3-b4c877182eb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:54.360616 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb49f2-d9ea-4705-96e3-b4c877182eb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:54.361060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb49f2-d9ea-4705-96e3-b4c877182eb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:54.362652 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb49f2-d9ea-4705-96e3-b4c877182eb8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:54.371683 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 71/150] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:58:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:58:54.372465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb49f2-d9ea-4705-96e3-b4c877182eb8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:54.418455 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:54.447284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62c533-45fb-4a41-bc11-e1bf9abafba2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:54.469865 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb49f2-d9ea-4705-96e3-b4c877182eb8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:54.480434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1200a91-5aca-4c9a-a36b-f25598169e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:54.481037 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1200a91-5aca-4c9a-a36b-f25598169e22 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:54.481317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1200a91-5aca-4c9a-a36b-f25598169e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:54.481589 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1200a91-5aca-4c9a-a36b-f25598169e22 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:54.482247 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1200a91-5aca-4c9a-a36b-f25598169e22 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:54.482555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1200a91-5aca-4c9a-a36b-f25598169e22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:54.483969 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1200a91-5aca-4c9a-a36b-f25598169e22 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:54.491697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 80/151] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:58:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:58:54.491697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1200a91-5aca-4c9a-a36b-f25598169e22 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:54.496590 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:54.504496 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee60af-a2f2-4289-9410-b7608d2f36f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:54.505209 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee60af-a2f2-4289-9410-b7608d2f36f8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:54.505492 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee60af-a2f2-4289-9410-b7608d2f36f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:54.505762 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee60af-a2f2-4289-9410-b7608d2f36f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:54.506881 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee60af-a2f2-4289-9410-b7608d2f36f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:54.508342 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee60af-a2f2-4289-9410-b7608d2f36f8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:54.509930 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 72/152] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:58:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:54.510682 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee60af-a2f2-4289-9410-b7608d2f36f8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:54.594147 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1200a91-5aca-4c9a-a36b-f25598169e22 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:54.606295 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee60af-a2f2-4289-9410-b7608d2f36f8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:54.627835 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf3082-90d3-46df-8ca1-6132124ec22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:54.628713 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf3082-90d3-46df-8ca1-6132124ec22c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:54.629000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf3082-90d3-46df-8ca1-6132124ec22c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:54.629911 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf3082-90d3-46df-8ca1-6132124ec22c None admin] PATH_INFO: `/v3/users/2f5decae62924e8fb9488297a6c89a58` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:54.631789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:54.639830 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4a395-87f8-4ae2-a15a-c186da0edd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:54.640456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4a395-87f8-4ae2-a15a-c186da0edd94 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:54.640737 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4a395-87f8-4ae2-a15a-c186da0edd94 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:54.641010 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4a395-87f8-4ae2-a15a-c186da0edd94 None admin] PATH_INFO: `/v3/users/2f5decae62924e8fb9488297a6c89a58` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:54.645850 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf3082-90d3-46df-8ca1-6132124ec22c None admin] RBAC: Authorizing `identity:get_user(user_id=2f5decae62924e8fb9488297a6c89a58)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:54.647327 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf3082-90d3-46df-8ca1-6132124ec22c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:54.648939 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 81/153] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:58:54 2022] GET /identity/v3/users/2f5decae62924e8fb9488297a6c89a58 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:54.649679 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf3082-90d3-46df-8ca1-6132124ec22c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:54.659944 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4a395-87f8-4ae2-a15a-c186da0edd94 None admin] RBAC: Authorizing `identity:get_user(user_id=2f5decae62924e8fb9488297a6c89a58)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:54.662468 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4a395-87f8-4ae2-a15a-c186da0edd94 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:54.664152 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 73/154] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:58:54 2022] GET /identity/v3/users/2f5decae62924e8fb9488297a6c89a58 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:54.664914 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4a395-87f8-4ae2-a15a-c186da0edd94 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:54.718455 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:54.747460 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf3082-90d3-46df-8ca1-6132124ec22c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:54.763935 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4a395-87f8-4ae2-a15a-c186da0edd94 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:54.781373 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852048d-8873-436c-a054-32b90723b04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:54.782087 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852048d-8873-436c-a054-32b90723b04c None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:54.782433 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852048d-8873-436c-a054-32b90723b04c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:54.782776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852048d-8873-436c-a054-32b90723b04c None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:54.783524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852048d-8873-436c-a054-32b90723b04c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:54.784959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2852048d-8873-436c-a054-32b90723b04c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:54.797586 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659138b-6d13-4f2d-aa22-cafb2e1b7633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:54.798251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659138b-6d13-4f2d-aa22-cafb2e1b7633 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:54.798585 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659138b-6d13-4f2d-aa22-cafb2e1b7633 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:54.798840 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659138b-6d13-4f2d-aa22-cafb2e1b7633 None admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:54.800065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659138b-6d13-4f2d-aa22-cafb2e1b7633 None admin] RBAC: Authorizing `identity:get_project(project_id=615d529fe91a4c82b4d4fe25f10a0478)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:54.801852 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659138b-6d13-4f2d-aa22-cafb2e1b7633 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:54.803504 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 74/155] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:58:54 2022] GET /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:54.808323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659138b-6d13-4f2d-aa22-cafb2e1b7633 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:54.808425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 82/156] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 02:58:54 2022] POST /identity/v3/users => generated 297 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:58:54.809575 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852048d-8873-436c-a054-32b90723b04c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:54.870298 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:54.922423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659138b-6d13-4f2d-aa22-cafb2e1b7633 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:54.933370 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852048d-8873-436c-a054-32b90723b04c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:54.947513 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:54.960556 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a307d0a-ad6e-44cd-9520-4be1f5bec3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:54.961211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a307d0a-ad6e-44cd-9520-4be1f5bec3ed None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:54.961568 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a307d0a-ad6e-44cd-9520-4be1f5bec3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:54.961746 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a307d0a-ad6e-44cd-9520-4be1f5bec3ed None admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:54.962920 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a307d0a-ad6e-44cd-9520-4be1f5bec3ed None admin] RBAC: Authorizing `identity:get_project(project_id=615d529fe91a4c82b4d4fe25f10a0478)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:54.964442 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a307d0a-ad6e-44cd-9520-4be1f5bec3ed None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:54.966297 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 75/157] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:58:54 2022] GET /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:54.972107 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a307d0a-ad6e-44cd-9520-4be1f5bec3ed None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:54.975415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd9ea8-0c7f-4b80-9102-8d9ad3a3e4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:54.976092 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd9ea8-0c7f-4b80-9102-8d9ad3a3e4f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:54.976373 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd9ea8-0c7f-4b80-9102-8d9ad3a3e4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:54.976646 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd9ea8-0c7f-4b80-9102-8d9ad3a3e4f3 None admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478/users/2f5decae62924e8fb9488297a6c89a58/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:54.978620 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd9ea8-0c7f-4b80-9102-8d9ad3a3e4f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=615d529fe91a4c82b4d4fe25f10a0478, user_id=2f5decae62924e8fb9488297a6c89a58, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:54.980124 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd9ea8-0c7f-4b80-9102-8d9ad3a3e4f3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:54.999640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 83/158] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 02:58:54 2022] PUT /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478/users/2f5decae62924e8fb9488297a6c89a58/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:58:55.114657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a307d0a-ad6e-44cd-9520-4be1f5bec3ed None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:55.157548 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b6dab-7bc9-46d8-9661-7e562a799106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:55.158157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b6dab-7bc9-46d8-9661-7e562a799106 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:55.158437 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b6dab-7bc9-46d8-9661-7e562a799106 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:55.158710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b6dab-7bc9-46d8-9661-7e562a799106 None admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478/users/2f5decae62924e8fb9488297a6c89a58/roles/baf207ce98e448f1a2e10c3310a0e33b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:55.161396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b6dab-7bc9-46d8-9661-7e562a799106 None admin] RBAC: Authorizing `identity:create_grant(project_id=615d529fe91a4c82b4d4fe25f10a0478, user_id=2f5decae62924e8fb9488297a6c89a58, role_id=baf207ce98e448f1a2e10c3310a0e33b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:55.162917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b6dab-7bc9-46d8-9661-7e562a799106 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:55.180242 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 76/159] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 02:58:54 2022] PUT /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478/users/2f5decae62924e8fb9488297a6c89a58/roles/baf207ce98e448f1a2e10c3310a0e33b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 02:58:57.099412 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a67545-3ec3-4072-870e-e0c03e2da8a0 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:57.099965 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a67545-3ec3-4072-870e-e0c03e2da8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:57.100240 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a67545-3ec3-4072-870e-e0c03e2da8a0 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:57.101706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 84/160] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:57.107206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e6912-646d-4285-8ebd-21ddd2dbd078 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:57.107668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e6912-646d-4285-8ebd-21ddd2dbd078 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:57.107739 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e6912-646d-4285-8ebd-21ddd2dbd078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:57.128100 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-548e6912-646d-4285-8ebd-21ddd2dbd078 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:57.129663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-548e6912-646d-4285-8ebd-21ddd2dbd078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:57.186661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:57.187414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025daad-3213-44e0-b289-ed291fa1344a None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:57.187898 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025daad-3213-44e0-b289-ed291fa1344a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:57.188446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025daad-3213-44e0-b289-ed291fa1344a None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:57.190043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 85/161] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:57.197180 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e81efb-4e74-4aee-b2d4-9fe5fa257c4a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:57.197769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e81efb-4e74-4aee-b2d4-9fe5fa257c4a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:57.198264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e81efb-4e74-4aee-b2d4-9fe5fa257c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:57.211658 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 77/162] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:58:57.222162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f3e81efb-4e74-4aee-b2d4-9fe5fa257c4a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:57.223815 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f3e81efb-4e74-4aee-b2d4-9fe5fa257c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:57.252586 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de482f44-89f9-4ce1-bb21-6da9acf42a1b None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:57.252925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de482f44-89f9-4ce1-bb21-6da9acf42a1b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:57.253149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de482f44-89f9-4ce1-bb21-6da9acf42a1b None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:57.255875 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 78/163] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 02:58:57.261900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4ee20-eee3-4b5e-9bbb-22a0ebef0d23 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:57.262201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4ee20-eee3-4b5e-9bbb-22a0ebef0d23 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:57.262460 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4ee20-eee3-4b5e-9bbb-22a0ebef0d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:57.282967 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-7df4ee20-eee3-4b5e-9bbb-22a0ebef0d23 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:57.286909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7df4ee20-eee3-4b5e-9bbb-22a0ebef0d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:57.303601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 86/164] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:58:57.355526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 79/165] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:58:57.358510 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3261d-667d-420f-88a2-62b10cc6b5a3 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:57.358792 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3261d-667d-420f-88a2-62b10cc6b5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:57.359103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3261d-667d-420f-88a2-62b10cc6b5a3 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:57.360431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 87/166] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:58:57.365719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba2384-a26c-4960-93ee-a7eafb8128b7 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:57.366079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba2384-a26c-4960-93ee-a7eafb8128b7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:57.366354 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba2384-a26c-4960-93ee-a7eafb8128b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:57.385210 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-29ba2384-a26c-4960-93ee-a7eafb8128b7 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:58:57.386373 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-29ba2384-a26c-4960-93ee-a7eafb8128b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:58:57.451650 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 80/167] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:58:57.492074 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3261d-667d-420f-88a2-62b10cc6b5a3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:57.533696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:57.596721 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3261d-667d-420f-88a2-62b10cc6b5a3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:57.612718 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba2384-a26c-4960-93ee-a7eafb8128b7 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:57.635756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd36d2-8622-4b0b-b9f4-11494cf64e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:57.636800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd36d2-8622-4b0b-b9f4-11494cf64e1a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:57.637084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd36d2-8622-4b0b-b9f4-11494cf64e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:57.637360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd36d2-8622-4b0b-b9f4-11494cf64e1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:57.653162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd36d2-8622-4b0b-b9f4-11494cf64e1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:58:57.658266 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:57.658581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-84bd36d2-8622-4b0b-b9f4-11494cf64e1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:58:57.658581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:57.658581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:57.658581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:57.658581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:57.658581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:57.658581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:57.658581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:57.658581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:57.658581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:57.658581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:57.658581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:57.658581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:58:57.658581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:58:57.658581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:58:57.658581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:58:57.659905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:58:57.659905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:58:57.659905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:58:57.659905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:58:57.659905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:57.659905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:57.659905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:57.659905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:57.659905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:57.659905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:57.659905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:57.659905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:57.659905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:57.659905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:57.659905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:57.659905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:57.661084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:57.661084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:57.661084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:57.661084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:57.661084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:58:57.661084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:58:57.661084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:58:57.661084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:58:57.661084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:58:57.661084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:58:57.661084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:58:57.661084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:58:57.661084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 88/168] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:58:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:58:57.662071 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd36d2-8622-4b0b-b9f4-11494cf64e1a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:57.724317 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba2384-a26c-4960-93ee-a7eafb8128b7 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:57.758912 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984a44f-8354-4319-b2a2-49e3409ae970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:57.759546 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984a44f-8354-4319-b2a2-49e3409ae970 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:57.759825 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984a44f-8354-4319-b2a2-49e3409ae970 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:57.760096 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984a44f-8354-4319-b2a2-49e3409ae970 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:57.766053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:57.770578 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd36d2-8622-4b0b-b9f4-11494cf64e1a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:57.770888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a984a44f-8354-4319-b2a2-49e3409ae970 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:58:57.775674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-a984a44f-8354-4319-b2a2-49e3409ae970 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:58:57.775674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:57.775674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:57.775674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:57.775674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:57.775674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:57.775674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:57.775674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:57.775674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:57.775674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:57.775674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:57.775674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:57.775674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:58:57.775674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:58:57.775674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:58:57.775674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:58:57.776719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:58:57.776719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:58:57.776719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:58:57.776719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:58:57.776719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:57.776719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:57.776719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:57.776719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:57.776719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:57.776719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:57.776719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:57.776719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:57.776719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:57.776719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:57.776719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:57.776719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:57.778153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:57.778153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:57.778153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:57.778153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:57.778153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:58:57.778153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:58:57.778153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:58:57.778153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:58:57.778153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:58:57.778153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:58:57.778153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:58:57.778153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:58:57.778153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 81/169] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 02:58:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:58:57.778153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984a44f-8354-4319-b2a2-49e3409ae970 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:57.806707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63527ab-aebe-4d11-8f2d-c833ead99252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:57.807373 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63527ab-aebe-4d11-8f2d-c833ead99252 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:57.807655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63527ab-aebe-4d11-8f2d-c833ead99252 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:57.807929 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63527ab-aebe-4d11-8f2d-c833ead99252 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:57.809044 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63527ab-aebe-4d11-8f2d-c833ead99252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:57.810486 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63527ab-aebe-4d11-8f2d-c833ead99252 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:57.812136 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 89/170] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:58:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:57.812895 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63527ab-aebe-4d11-8f2d-c833ead99252 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:57.878141 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:57.881508 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984a44f-8354-4319-b2a2-49e3409ae970 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:57.911314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63527ab-aebe-4d11-8f2d-c833ead99252 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:57.915069 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9871aeb-80fe-4b9e-b7bb-3e262bfa1fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:57.915674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9871aeb-80fe-4b9e-b7bb-3e262bfa1fea None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:57.915952 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9871aeb-80fe-4b9e-b7bb-3e262bfa1fea None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:57.916223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9871aeb-80fe-4b9e-b7bb-3e262bfa1fea None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:57.916886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9871aeb-80fe-4b9e-b7bb-3e262bfa1fea None admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:57.917187 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9871aeb-80fe-4b9e-b7bb-3e262bfa1fea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:57.918580 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9871aeb-80fe-4b9e-b7bb-3e262bfa1fea None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:57.927338 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 82/171] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:58:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:57.928211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9871aeb-80fe-4b9e-b7bb-3e262bfa1fea None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:57.949597 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082f66c-e0d4-4672-996c-fa08fd4b334b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:57.950234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082f66c-e0d4-4672-996c-fa08fd4b334b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:57.950515 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082f66c-e0d4-4672-996c-fa08fd4b334b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:57.950786 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082f66c-e0d4-4672-996c-fa08fd4b334b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:57.961382 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082f66c-e0d4-4672-996c-fa08fd4b334b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:58:57.966179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-7082f66c-e0d4-4672-996c-fa08fd4b334b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:58:57.966179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:58:57.966179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:58:57.966179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:58:57.966179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:58:57.966179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:58:57.966179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:58:57.966179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:58:57.966179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:58:57.966179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:58:57.966179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:58:57.966179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:58:57.966179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:58:57.966179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:58:57.966179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:58:57.966179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:58:57.967164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:58:57.967164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:58:57.967164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:58:57.967164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:58:57.967164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:58:57.967164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:58:57.967164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:58:57.967164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:58:57.967164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:58:57.967164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:58:57.967164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:58:57.967164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:58:57.967164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:58:57.967164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:58:57.967164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:58:57.967164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:58:57.968161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:58:57.968161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:58:57.968161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:58:57.968161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:58:57.968161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:58:57.968161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:58:57.968161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:58:57.968161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:58:57.968161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:58:57.968161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:58:57.968161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:58:57.968161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:58:57.968161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 90/172] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Oct 17 02:58:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:58:57.968161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082f66c-e0d4-4672-996c-fa08fd4b334b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:57.996239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:58.027486 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9871aeb-80fe-4b9e-b7bb-3e262bfa1fea None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:58.062627 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58c15b-68e4-4e20-ad02-ed59d831ba52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:58.063266 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58c15b-68e4-4e20-ad02-ed59d831ba52 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:58.063550 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58c15b-68e4-4e20-ad02-ed59d831ba52 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:58.063826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58c15b-68e4-4e20-ad02-ed59d831ba52 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:58.064499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58c15b-68e4-4e20-ad02-ed59d831ba52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:58.064807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58c15b-68e4-4e20-ad02-ed59d831ba52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:58.066190 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58c15b-68e4-4e20-ad02-ed59d831ba52 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:58.068178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082f66c-e0d4-4672-996c-fa08fd4b334b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:58.074173 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 83/173] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:58:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:58.075792 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58c15b-68e4-4e20-ad02-ed59d831ba52 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:58.100749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:58.101574 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b6760-44ee-45fb-841f-a064b093db7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:58.102477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b6760-44ee-45fb-841f-a064b093db7d None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:58.102771 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b6760-44ee-45fb-841f-a064b093db7d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:58.103136 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b6760-44ee-45fb-841f-a064b093db7d None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:58.103797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b6760-44ee-45fb-841f-a064b093db7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:58.105265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b6760-44ee-45fb-841f-a064b093db7d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:58.126082 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 91/174] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 02:58:57 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:58:58.126907 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b6760-44ee-45fb-841f-a064b093db7d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:58.199981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58c15b-68e4-4e20-ad02-ed59d831ba52 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:58.222163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:58.242605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c8d1d-4fae-437f-bfe2-3a725de17601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:58.243301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c8d1d-4fae-437f-bfe2-3a725de17601 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:58.243642 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c8d1d-4fae-437f-bfe2-3a725de17601 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:58.243847 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c8d1d-4fae-437f-bfe2-3a725de17601 None admin] PATH_INFO: `/v3/users/95372ca150c549988107a53899562902` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:58.247458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b6760-44ee-45fb-841f-a064b093db7d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:58.259970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c8d1d-4fae-437f-bfe2-3a725de17601 None admin] RBAC: Authorizing `identity:get_user(user_id=95372ca150c549988107a53899562902)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:58.261996 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c8d1d-4fae-437f-bfe2-3a725de17601 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:58.264139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 84/175] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:58:58 2022] GET /identity/v3/users/95372ca150c549988107a53899562902 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:58.265369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c8d1d-4fae-437f-bfe2-3a725de17601 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:58.294842 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bf142-1106-49f5-bffa-ecc4f49ae60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:58.295507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bf142-1106-49f5-bffa-ecc4f49ae60a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:58.295892 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bf142-1106-49f5-bffa-ecc4f49ae60a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:58.296066 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bf142-1106-49f5-bffa-ecc4f49ae60a None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:58.296752 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566bf142-1106-49f5-bffa-ecc4f49ae60a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:58.297307 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566bf142-1106-49f5-bffa-ecc4f49ae60a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:58.298473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566bf142-1106-49f5-bffa-ecc4f49ae60a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:58.312933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 92/176] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:58:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:58.312933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bf142-1106-49f5-bffa-ecc4f49ae60a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:58.367917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:58.388238 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c8d1d-4fae-437f-bfe2-3a725de17601 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:58.431799 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269feb3c-77a2-4c25-a3fd-7339d91d0d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:58.432441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269feb3c-77a2-4c25-a3fd-7339d91d0d8c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:58.432716 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269feb3c-77a2-4c25-a3fd-7339d91d0d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:58.433000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269feb3c-77a2-4c25-a3fd-7339d91d0d8c None admin] PATH_INFO: `/v3/users/2f5decae62924e8fb9488297a6c89a58` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:58.436648 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bf142-1106-49f5-bffa-ecc4f49ae60a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:58.436914 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269feb3c-77a2-4c25-a3fd-7339d91d0d8c None admin] RBAC: Authorizing `identity:get_user(user_id=2f5decae62924e8fb9488297a6c89a58)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:58.438391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269feb3c-77a2-4c25-a3fd-7339d91d0d8c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:58.440397 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 85/177] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:58:58 2022] GET /identity/v3/users/2f5decae62924e8fb9488297a6c89a58 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:58.440960 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269feb3c-77a2-4c25-a3fd-7339d91d0d8c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:58.483693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254631a5-d674-4967-845b-8e4cfbd87a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:58.484272 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254631a5-d674-4967-845b-8e4cfbd87a76 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:58.484559 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254631a5-d674-4967-845b-8e4cfbd87a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:58.484838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254631a5-d674-4967-845b-8e4cfbd87a76 None admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:58.486260 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254631a5-d674-4967-845b-8e4cfbd87a76 None admin] RBAC: Authorizing `identity:get_project(project_id=615d529fe91a4c82b4d4fe25f10a0478)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:58.487745 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254631a5-d674-4967-845b-8e4cfbd87a76 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:58.491040 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 93/178] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:58:58 2022] GET /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:58.491172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254631a5-d674-4967-845b-8e4cfbd87a76 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:58.508021 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:58.561057 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269feb3c-77a2-4c25-a3fd-7339d91d0d8c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:58.601258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6cb55-e0f9-4892-83ab-7e95e31fee0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:58.601900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6cb55-e0f9-4892-83ab-7e95e31fee0c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:58.602180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6cb55-e0f9-4892-83ab-7e95e31fee0c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:58.602554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6cb55-e0f9-4892-83ab-7e95e31fee0c None admin] PATH_INFO: `/v3/users/2f5decae62924e8fb9488297a6c89a58` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:58.603896 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6cb55-e0f9-4892-83ab-7e95e31fee0c None admin] RBAC: Authorizing `identity:get_user(user_id=2f5decae62924e8fb9488297a6c89a58)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:58.605374 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6cb55-e0f9-4892-83ab-7e95e31fee0c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:58.606045 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254631a5-d674-4967-845b-8e4cfbd87a76 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:58.607142 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 86/179] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:58:58 2022] GET /identity/v3/users/2f5decae62924e8fb9488297a6c89a58 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:58.612100 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6cb55-e0f9-4892-83ab-7e95e31fee0c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:58.633276 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:58.646812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd24e7-28d4-4390-8bbb-51d90a443c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:58.647751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd24e7-28d4-4390-8bbb-51d90a443c24 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:58.648198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd24e7-28d4-4390-8bbb-51d90a443c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:58.648619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd24e7-28d4-4390-8bbb-51d90a443c24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:58.721313 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6cb55-e0f9-4892-83ab-7e95e31fee0c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:58.723863 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd24e7-28d4-4390-8bbb-51d90a443c24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:58.725493 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd24e7-28d4-4390-8bbb-51d90a443c24 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:58.740464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:58.760331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06233648-083a-46e3-b4ee-d645690e88a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:58.761095 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06233648-083a-46e3-b4ee-d645690e88a5 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:58.761375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06233648-083a-46e3-b4ee-d645690e88a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:58.761646 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06233648-083a-46e3-b4ee-d645690e88a5 None admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478/users/95372ca150c549988107a53899562902/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:58.763689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06233648-083a-46e3-b4ee-d645690e88a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=615d529fe91a4c82b4d4fe25f10a0478, user_id=95372ca150c549988107a53899562902, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:58.765252 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06233648-083a-46e3-b4ee-d645690e88a5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:58.779415 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 87/180] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 02:58:58 2022] PUT /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478/users/95372ca150c549988107a53899562902/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:58:58.783917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06233648-083a-46e3-b4ee-d645690e88a5 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:58.848995 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 94/181] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:58:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 02:58:58.856472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd24e7-28d4-4390-8bbb-51d90a443c24 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:58.965254 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:59.008364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd24e7-28d4-4390-8bbb-51d90a443c24 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:59.009652 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06233648-083a-46e3-b4ee-d645690e88a5 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:59.022854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:59.048724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8eb582-d855-43be-8d3b-374e9f2db0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:59.049958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8eb582-d855-43be-8d3b-374e9f2db0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:59.050469 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8eb582-d855-43be-8d3b-374e9f2db0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:59.050697 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b3e54-958b-4834-864a-a2c70bb0bad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:59.051768 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8eb582-d855-43be-8d3b-374e9f2db0c7 None admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:59.051939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b3e54-958b-4834-864a-a2c70bb0bad3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:59.052179 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b3e54-958b-4834-864a-a2c70bb0bad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:59.052451 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b3e54-958b-4834-864a-a2c70bb0bad3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:59.053745 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8eb582-d855-43be-8d3b-374e9f2db0c7 None admin] RBAC: Authorizing `identity:get_project(project_id=615d529fe91a4c82b4d4fe25f10a0478)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:59.055488 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8eb582-d855-43be-8d3b-374e9f2db0c7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:59.057360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 95/182] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:58:58 2022] GET /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:58:59.063098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8eb582-d855-43be-8d3b-374e9f2db0c7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:59.150827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b3e54-958b-4834-864a-a2c70bb0bad3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:59.152498 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b3e54-958b-4834-864a-a2c70bb0bad3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:59.188019 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8eb582-d855-43be-8d3b-374e9f2db0c7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:59.228316 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64871f1-4f53-474c-aff0-277c5b06c1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:59.229308 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64871f1-4f53-474c-aff0-277c5b06c1af None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:59.230566 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64871f1-4f53-474c-aff0-277c5b06c1af None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:59.231076 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64871f1-4f53-474c-aff0-277c5b06c1af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:59.232025 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64871f1-4f53-474c-aff0-277c5b06c1af None admin] RBAC: Adding query filter params (role.id=ea0fd0c10f2f4e529270edbe3525d4c6, scope.project.id=615d529fe91a4c82b4d4fe25f10a0478, user.id=2f5decae62924e8fb9488297a6c89a58) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:59.232548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64871f1-4f53-474c-aff0-277c5b06c1af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:59.234217 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64871f1-4f53-474c-aff0-277c5b06c1af None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:59.242223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 96/183] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 02:58:59 2022] GET /identity/v3/role_assignments?user.id=2f5decae62924e8fb9488297a6c89a58&scope.project.id=615d529fe91a4c82b4d4fe25f10a0478&role.id=ea0fd0c10f2f4e529270edbe3525d4c6 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:58:59.309375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 88/184] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:58:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 02:58:59.311249 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:59.315241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64871f1-4f53-474c-aff0-277c5b06c1af None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:59.372007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:58:59.430823 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64871f1-4f53-474c-aff0-277c5b06c1af None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:59.466658 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357283be-d295-4ab0-a79e-e79c2f332ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:59.467347 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357283be-d295-4ab0-a79e-e79c2f332ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:59.467630 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357283be-d295-4ab0-a79e-e79c2f332ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:59.467908 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357283be-d295-4ab0-a79e-e79c2f332ad5 None admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:59.469085 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357283be-d295-4ab0-a79e-e79c2f332ad5 None admin] RBAC: Authorizing `identity:get_project(project_id=615d529fe91a4c82b4d4fe25f10a0478)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:59.470640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357283be-d295-4ab0-a79e-e79c2f332ad5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:59.472710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 97/185] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:58:59 2022] GET /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:58:59.478709 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b3e54-958b-4834-864a-a2c70bb0bad3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:58:59.590567 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b3e54-958b-4834-864a-a2c70bb0bad3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:58:59.632654 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab24995-ab8c-4f64-86e6-0fbf343b4add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:58:59.633264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab24995-ab8c-4f64-86e6-0fbf343b4add None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:58:59.633543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab24995-ab8c-4f64-86e6-0fbf343b4add None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:58:59.633867 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab24995-ab8c-4f64-86e6-0fbf343b4add None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:58:59.634602 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab24995-ab8c-4f64-86e6-0fbf343b4add None admin] RBAC: Adding query filter params (role.id=baf207ce98e448f1a2e10c3310a0e33b, scope.project.id=615d529fe91a4c82b4d4fe25f10a0478, user.id=2f5decae62924e8fb9488297a6c89a58) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:58:59.634990 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab24995-ab8c-4f64-86e6-0fbf343b4add None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:58:59.636451 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab24995-ab8c-4f64-86e6-0fbf343b4add None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:58:59.643647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 89/186] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 02:58:59 2022] GET /identity/v3/role_assignments?user.id=2f5decae62924e8fb9488297a6c89a58&scope.project.id=615d529fe91a4c82b4d4fe25f10a0478&role.id=baf207ce98e448f1a2e10c3310a0e33b => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:00.516333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b18b12-a14f-4e66-925a-ede88db5bfc1 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:00.516831 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b18b12-a14f-4e66-925a-ede88db5bfc1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:00.516912 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b18b12-a14f-4e66-925a-ede88db5bfc1 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:00.518387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 98/187] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:00.523777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285cd78-a8b3-43b8-adfe-11d453b15115 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:00.524175 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285cd78-a8b3-43b8-adfe-11d453b15115 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:00.524332 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285cd78-a8b3-43b8-adfe-11d453b15115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:00.543721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5285cd78-a8b3-43b8-adfe-11d453b15115 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:00.544896 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5285cd78-a8b3-43b8-adfe-11d453b15115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:00.580607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:00.614550 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 90/188] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:00.787875 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be69123-22f7-41ab-ab63-8da03a28a417 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:00.788202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be69123-22f7-41ab-ab63-8da03a28a417 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:00.788441 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be69123-22f7-41ab-ab63-8da03a28a417 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:00.789924 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 99/189] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:00.795000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe02e4-4191-419d-9a79-1f67a67785e6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:00.795242 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe02e4-4191-419d-9a79-1f67a67785e6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:00.795519 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe02e4-4191-419d-9a79-1f67a67785e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:00.814326 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-32fe02e4-4191-419d-9a79-1f67a67785e6 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:00.815535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-32fe02e4-4191-419d-9a79-1f67a67785e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:00.882620 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 91/190] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:00.886577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be69123-22f7-41ab-ab63-8da03a28a417 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:00.989562 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be69123-22f7-41ab-ab63-8da03a28a417 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:01.002299 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:01.026783 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5694bf-9724-42de-998f-c0817dbd71aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:01.027446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5694bf-9724-42de-998f-c0817dbd71aa None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:01.027730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5694bf-9724-42de-998f-c0817dbd71aa None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:01.028003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5694bf-9724-42de-998f-c0817dbd71aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:01.038774 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5694bf-9724-42de-998f-c0817dbd71aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:01.043735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-5e5694bf-9724-42de-998f-c0817dbd71aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 02:59:01.043735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:01.043735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:01.043735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:01.043735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:01.043735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:01.043735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:01.043735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:01.043735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:01.043735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:01.043735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:01.043735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:01.043735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:01.043735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:01.043735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:01.043735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:01.044619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:01.044619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:01.044619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:01.044619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:01.044619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:01.044619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:01.044619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:01.044619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:01.044619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:01.044619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:01.044619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:01.044619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:01.044619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:01.044619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:01.044619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:01.044619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:01.045477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:01.045477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:01.045477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:01.045477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:01.045477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:01.045477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:01.045477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:01.045477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:01.045477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:01.045477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:01.045477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 02:59:01.045477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:01.045477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 100/191] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:01.052349 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe02e4-4191-419d-9a79-1f67a67785e6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:01.154498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5694bf-9724-42de-998f-c0817dbd71aa None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:01.158638 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe02e4-4191-419d-9a79-1f67a67785e6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:01.160219 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:01.196684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f9530-d3c9-4306-8d4a-e4bc83949d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:01.197297 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f9530-d3c9-4306-8d4a-e4bc83949d11 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:01.197581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f9530-d3c9-4306-8d4a-e4bc83949d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:01.197884 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f9530-d3c9-4306-8d4a-e4bc83949d11 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:01.198544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f9530-d3c9-4306-8d4a-e4bc83949d11 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:01.198847 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f9530-d3c9-4306-8d4a-e4bc83949d11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:01.200274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f9530-d3c9-4306-8d4a-e4bc83949d11 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:01.208675 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 92/192] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:59:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:01.209833 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f94b9-e647-4e18-a9b8-8b3cdd50bd00 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:01.210123 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f94b9-e647-4e18-a9b8-8b3cdd50bd00 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:01.210399 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f94b9-e647-4e18-a9b8-8b3cdd50bd00 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:01.211748 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 93/193] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:01.215931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f94b9-e647-4e18-a9b8-8b3cdd50bd00 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:01.261155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5694bf-9724-42de-998f-c0817dbd71aa None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:01.296641 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70582ee3-3e60-4ffc-b3e6-a0efbd655336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:01.297206 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70582ee3-3e60-4ffc-b3e6-a0efbd655336 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:01.297481 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70582ee3-3e60-4ffc-b3e6-a0efbd655336 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:01.297755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70582ee3-3e60-4ffc-b3e6-a0efbd655336 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:01.298847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70582ee3-3e60-4ffc-b3e6-a0efbd655336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:01.300301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70582ee3-3e60-4ffc-b3e6-a0efbd655336 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:01.302602 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 101/194] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:01.303724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02035998-5433-4acd-b31c-25e7f73ca507 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:01.304001 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02035998-5433-4acd-b31c-25e7f73ca507 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:01.304272 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02035998-5433-4acd-b31c-25e7f73ca507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:01.322149 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-02035998-5433-4acd-b31c-25e7f73ca507 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:01.323385 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-02035998-5433-4acd-b31c-25e7f73ca507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:01.323680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f94b9-e647-4e18-a9b8-8b3cdd50bd00 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:01.356420 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dada1f-06aa-46e4-9017-3be97ed1cdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:01.357047 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dada1f-06aa-46e4-9017-3be97ed1cdc6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:01.357318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dada1f-06aa-46e4-9017-3be97ed1cdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:01.357593 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dada1f-06aa-46e4-9017-3be97ed1cdc6 None admin] PATH_INFO: `/v3/users/3cb0877a442a446c806fda81718c6642` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:01.357695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:01.371073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dada1f-06aa-46e4-9017-3be97ed1cdc6 None admin] RBAC: Authorizing `identity:get_user(user_id=3cb0877a442a446c806fda81718c6642)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:01.372550 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dada1f-06aa-46e4-9017-3be97ed1cdc6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:01.374144 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 94/195] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:59:01 2022] GET /identity/v3/users/3cb0877a442a446c806fda81718c6642 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:01.375676 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dada1f-06aa-46e4-9017-3be97ed1cdc6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:01.391356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 102/196] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:01.392445 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02035998-5433-4acd-b31c-25e7f73ca507 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:01.400760 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:01.492078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dada1f-06aa-46e4-9017-3be97ed1cdc6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:01.496572 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02035998-5433-4acd-b31c-25e7f73ca507 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:01.530540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0daa6d-37af-40f6-bfc4-34d45352270e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:01.531154 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46560a-b791-45ed-bbfe-b52513beec45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:01.531494 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0daa6d-37af-40f6-bfc4-34d45352270e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:01.531600 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46560a-b791-45ed-bbfe-b52513beec45 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:01.531680 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0daa6d-37af-40f6-bfc4-34d45352270e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:01.531747 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46560a-b791-45ed-bbfe-b52513beec45 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:01.531816 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0daa6d-37af-40f6-bfc4-34d45352270e None admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:01.531940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46560a-b791-45ed-bbfe-b52513beec45 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:01.532743 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46560a-b791-45ed-bbfe-b52513beec45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:01.533093 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0daa6d-37af-40f6-bfc4-34d45352270e None admin] RBAC: Authorizing `identity:get_project(project_id=615d529fe91a4c82b4d4fe25f10a0478)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:01.534179 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46560a-b791-45ed-bbfe-b52513beec45 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:01.534663 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0daa6d-37af-40f6-bfc4-34d45352270e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:01.536667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 103/197] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:59:01 2022] GET /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:01.542581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0daa6d-37af-40f6-bfc4-34d45352270e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:01.555703 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 95/198] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 02:59:01 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:59:01.590578 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:01.658075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0daa6d-37af-40f6-bfc4-34d45352270e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:01.671176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46560a-b791-45ed-bbfe-b52513beec45 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:01.696728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94270b60-677e-4842-aa08-6714f72a53dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:01.697404 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94270b60-677e-4842-aa08-6714f72a53dd None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:01.697680 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94270b60-677e-4842-aa08-6714f72a53dd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:01.697951 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94270b60-677e-4842-aa08-6714f72a53dd None admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478/users/3cb0877a442a446c806fda81718c6642/roles/1dc8aea1b26145879f699bdeb2f077da` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:01.699936 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94270b60-677e-4842-aa08-6714f72a53dd None admin] RBAC: Authorizing `identity:create_grant(project_id=615d529fe91a4c82b4d4fe25f10a0478, user_id=3cb0877a442a446c806fda81718c6642, role_id=1dc8aea1b26145879f699bdeb2f077da)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:01.701412 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94270b60-677e-4842-aa08-6714f72a53dd None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:01.717163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 104/199] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 02:59:01 2022] PUT /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478/users/3cb0877a442a446c806fda81718c6642/roles/1dc8aea1b26145879f699bdeb2f077da => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 02:59:01.773872 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:01.801896 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46560a-b791-45ed-bbfe-b52513beec45 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:01.843909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960b9ed-fb97-4275-a727-f2d2ae960094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:01.844600 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960b9ed-fb97-4275-a727-f2d2ae960094 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:01.844974 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960b9ed-fb97-4275-a727-f2d2ae960094 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:01.845256 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960b9ed-fb97-4275-a727-f2d2ae960094 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:01.859322 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960b9ed-fb97-4275-a727-f2d2ae960094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:01.865742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-3960b9ed-fb97-4275-a727-f2d2ae960094 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:59:01.865742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:01.865742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:01.865742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:01.865742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:01.865742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:01.865742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:01.865742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:01.865742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:01.865742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:01.865742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:01.865742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:01.865742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:01.865742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:01.865742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:01.865742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:01.866700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:01.866700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:01.866700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:01.866700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:01.866700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:01.866700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:01.866700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:01.866700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:01.866700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:01.866700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:01.866700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:01.866700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:01.866700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:01.866700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:01.866700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:01.866700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:01.867635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:01.867635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:01.867635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:01.867635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:01.867635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:01.867635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:01.867635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:01.867635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:01.867635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:01.867635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:01.867635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:59:01.867635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:01.867635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 96/200] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:59:01.875360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94270b60-677e-4842-aa08-6714f72a53dd None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:01.982941 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:01.987403 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94270b60-677e-4842-aa08-6714f72a53dd None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:02.024619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8dd471-97f2-4881-b5d2-8451e71cd1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:02.025621 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dd471-97f2-4881-b5d2-8451e71cd1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:02.026103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dd471-97f2-4881-b5d2-8451e71cd1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:02.026574 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8dd471-97f2-4881-b5d2-8451e71cd1c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:02.027480 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8dd471-97f2-4881-b5d2-8451e71cd1c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:02.027987 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8dd471-97f2-4881-b5d2-8451e71cd1c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:02.029583 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8dd471-97f2-4881-b5d2-8451e71cd1c7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:02.038466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 105/201] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:59:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:02.044491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960b9ed-fb97-4275-a727-f2d2ae960094 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:02.150036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960b9ed-fb97-4275-a727-f2d2ae960094 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:02.185104 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743e6ac-1d9a-4322-84eb-b3feafabe17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:02.185733 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743e6ac-1d9a-4322-84eb-b3feafabe17e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:02.186010 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743e6ac-1d9a-4322-84eb-b3feafabe17e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:02.186279 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743e6ac-1d9a-4322-84eb-b3feafabe17e None admin] PATH_INFO: `/v3/users/95372ca150c549988107a53899562902` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:02.187445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743e6ac-1d9a-4322-84eb-b3feafabe17e None admin] RBAC: Authorizing `identity:get_user(user_id=95372ca150c549988107a53899562902)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:02.188911 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743e6ac-1d9a-4322-84eb-b3feafabe17e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:02.190798 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 97/202] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:59:02 2022] GET /identity/v3/users/95372ca150c549988107a53899562902 => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:02.196623 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8dd471-97f2-4881-b5d2-8451e71cd1c7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:02.308463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8dd471-97f2-4881-b5d2-8451e71cd1c7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:02.346493 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0dd77-12a4-483f-a152-081b644af0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:02.347165 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0dd77-12a4-483f-a152-081b644af0ea None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:02.347445 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0dd77-12a4-483f-a152-081b644af0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:02.347715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0dd77-12a4-483f-a152-081b644af0ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:02.376322 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:02.418149 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca0dd77-12a4-483f-a152-081b644af0ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:02.419556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca0dd77-12a4-483f-a152-081b644af0ea None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:02.520724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 106/203] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:59:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 02:59:02.526277 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743e6ac-1d9a-4322-84eb-b3feafabe17e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:02.535076 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:02.634813 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743e6ac-1d9a-4322-84eb-b3feafabe17e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:02.671258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86deaf-2afb-49b9-9895-3d5fd1f24e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:02.671820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86deaf-2afb-49b9-9895-3d5fd1f24e7e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:02.672096 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86deaf-2afb-49b9-9895-3d5fd1f24e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:02.672363 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86deaf-2afb-49b9-9895-3d5fd1f24e7e None admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:02.673505 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86deaf-2afb-49b9-9895-3d5fd1f24e7e None admin] RBAC: Authorizing `identity:get_project(project_id=615d529fe91a4c82b4d4fe25f10a0478)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:02.675226 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86deaf-2afb-49b9-9895-3d5fd1f24e7e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:02.677547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 98/204] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:59:02 2022] GET /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:02.683186 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0dd77-12a4-483f-a152-081b644af0ea None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:02.745228 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:02.786292 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0dd77-12a4-483f-a152-081b644af0ea None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:02.820117 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8bae02-74e4-4e75-93bb-7e0dc25f7e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:02.820719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bae02-74e4-4e75-93bb-7e0dc25f7e9c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:02.820996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bae02-74e4-4e75-93bb-7e0dc25f7e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:02.821264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bae02-74e4-4e75-93bb-7e0dc25f7e9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:02.821984 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8bae02-74e4-4e75-93bb-7e0dc25f7e9c None admin] RBAC: Adding query filter params (role.id=11de9783bae3450ea53955de9ebcc9d1, scope.project.id=615d529fe91a4c82b4d4fe25f10a0478, user.id=95372ca150c549988107a53899562902) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:02.822287 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8bae02-74e4-4e75-93bb-7e0dc25f7e9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:02.823741 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8bae02-74e4-4e75-93bb-7e0dc25f7e9c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:02.835031 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 107/205] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 02:59:02 2022] GET /identity/v3/role_assignments?user.id=95372ca150c549988107a53899562902&scope.project.id=615d529fe91a4c82b4d4fe25f10a0478&role.id=11de9783bae3450ea53955de9ebcc9d1 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:03.917416 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dc04e-ff56-4fa7-856c-b16d4f830085 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:03.917935 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dc04e-ff56-4fa7-856c-b16d4f830085 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:03.918043 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dc04e-ff56-4fa7-856c-b16d4f830085 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:03.919522 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 99/206] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:03.924473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18b86a-9ce3-4fea-b460-0ca52dad3966 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:03.924846 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18b86a-9ce3-4fea-b460-0ca52dad3966 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:03.925025 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18b86a-9ce3-4fea-b460-0ca52dad3966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:03.944094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-da18b86a-9ce3-4fea-b460-0ca52dad3966 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:03.945504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-da18b86a-9ce3-4fea-b460-0ca52dad3966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:03.966942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf9ca1-ccfa-4bc4-8c46-06aac6de29f3 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:03.967234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf9ca1-ccfa-4bc4-8c46-06aac6de29f3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:03.967510 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf9ca1-ccfa-4bc4-8c46-06aac6de29f3 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:03.968850 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 100/207] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 02:59:03.975035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68aa0f2-45a0-488f-ba2a-2ae09b11d60d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:03.975326 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68aa0f2-45a0-488f-ba2a-2ae09b11d60d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:03.975605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68aa0f2-45a0-488f-ba2a-2ae09b11d60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:03.993810 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d68aa0f2-45a0-488f-ba2a-2ae09b11d60d None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:03.995036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d68aa0f2-45a0-488f-ba2a-2ae09b11d60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:04.016973 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 108/208] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:04.018222 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cc5cf-0492-4004-b653-beadce67f0f0 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:04.018511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cc5cf-0492-4004-b653-beadce67f0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:04.018784 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313cc5cf-0492-4004-b653-beadce67f0f0 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:04.020991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 109/209] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:04.026323 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da40af3-d02a-440d-928e-70017f245550 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:04.026757 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da40af3-d02a-440d-928e-70017f245550 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:04.027069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da40af3-d02a-440d-928e-70017f245550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:04.047256 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8da40af3-d02a-440d-928e-70017f245550 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:04.048448 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8da40af3-d02a-440d-928e-70017f245550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:04.063874 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 101/210] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:04.100669 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:04.118698 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 110/211] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:59:04.283529 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68aa0f2-45a0-488f-ba2a-2ae09b11d60d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:04.287838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:04.334550 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da40af3-d02a-440d-928e-70017f245550 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:04.388764 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68aa0f2-45a0-488f-ba2a-2ae09b11d60d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:04.423881 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c7b3b-d586-4564-b697-cb28f3def2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:04.424512 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c7b3b-d586-4564-b697-cb28f3def2cc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:04.424794 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c7b3b-d586-4564-b697-cb28f3def2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:04.425069 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c7b3b-d586-4564-b697-cb28f3def2cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:04.435085 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da40af3-d02a-440d-928e-70017f245550 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:04.435385 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c7b3b-d586-4564-b697-cb28f3def2cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:04.440183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-4f2c7b3b-d586-4564-b697-cb28f3def2cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:59:04.440183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:04.440183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:04.440183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:04.440183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:04.440183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:04.440183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:04.440183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:04.440183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:04.440183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:04.440183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:04.440183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:04.440183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:04.440183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:04.440183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:04.440183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:04.441275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:04.441275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:04.441275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:04.441275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:04.441275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:04.441275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:04.441275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:04.441275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:04.441275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:04.441275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:04.441275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:04.441275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:04.441275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:04.441275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:04.441275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:04.441275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:04.442702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:04.442702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:04.442702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:04.442702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:04.442702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:04.442702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:04.442702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:04.442702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:04.442702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:04.442702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:04.442702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:59:04.442702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:04.442702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 102/212] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:04.442702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c7b3b-d586-4564-b697-cb28f3def2cc None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:04.472738 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b802fe5-a27a-4c56-89ef-c86bfdd5e29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:04.473403 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b802fe5-a27a-4c56-89ef-c86bfdd5e29e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:04.473685 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b802fe5-a27a-4c56-89ef-c86bfdd5e29e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:04.473963 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b802fe5-a27a-4c56-89ef-c86bfdd5e29e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:04.483934 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b802fe5-a27a-4c56-89ef-c86bfdd5e29e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:04.489269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-0b802fe5-a27a-4c56-89ef-c86bfdd5e29e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 02:59:04.489269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:04.489269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:04.489269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:04.489269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:04.489269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:04.489269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:04.489269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:04.489269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:04.489269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:04.489269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:04.489269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:04.489269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:04.489269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:04.489269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:04.489269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:04.490301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:04.490301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:04.490301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:04.490301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:04.490301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:04.490301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:04.490301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:04.490301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:04.490301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:04.490301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:04.490301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:04.490301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:04.490301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:04.490301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:04.490301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:04.490301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:04.491320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:04.491320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:04.491320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:04.491320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:04.491320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:04.491320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:04.491320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:04.491320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:04.491320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:04.491320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:04.491320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 02:59:04.491320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:04.491320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 111/213] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:59:04.492179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b802fe5-a27a-4c56-89ef-c86bfdd5e29e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:04.509290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:04.537097 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:04.546896 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c7b3b-d586-4564-b697-cb28f3def2cc None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:04.581266 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5804d4-f09e-43c0-9de3-264ca6632a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:04.581878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5804d4-f09e-43c0-9de3-264ca6632a12 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:04.582152 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5804d4-f09e-43c0-9de3-264ca6632a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:04.582424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5804d4-f09e-43c0-9de3-264ca6632a12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:04.583545 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5804d4-f09e-43c0-9de3-264ca6632a12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:04.584983 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5804d4-f09e-43c0-9de3-264ca6632a12 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:04.586527 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 103/214] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:04.587306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5804d4-f09e-43c0-9de3-264ca6632a12 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:04.592879 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b802fe5-a27a-4c56-89ef-c86bfdd5e29e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:04.626999 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763b268-7270-4524-ab31-2909babe8787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:04.627603 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763b268-7270-4524-ab31-2909babe8787 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:04.627877 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763b268-7270-4524-ab31-2909babe8787 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:04.628143 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763b268-7270-4524-ab31-2909babe8787 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:04.628802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763b268-7270-4524-ab31-2909babe8787 None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:04.629926 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763b268-7270-4524-ab31-2909babe8787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:04.631324 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763b268-7270-4524-ab31-2909babe8787 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:04.641442 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 112/215] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:59:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:04.642181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763b268-7270-4524-ab31-2909babe8787 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:04.684191 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5804d4-f09e-43c0-9de3-264ca6632a12 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:04.713231 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:04.718604 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1a650-b1ab-4f3f-935d-1b6592acfdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:04.719249 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1a650-b1ab-4f3f-935d-1b6592acfdfd None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:04.719529 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1a650-b1ab-4f3f-935d-1b6592acfdfd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:04.719804 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1a650-b1ab-4f3f-935d-1b6592acfdfd None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:04.720469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1a650-b1ab-4f3f-935d-1b6592acfdfd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:04.720770 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1a650-b1ab-4f3f-935d-1b6592acfdfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:04.722134 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1a650-b1ab-4f3f-935d-1b6592acfdfd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:04.730454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 104/216] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:59:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:04.731259 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1a650-b1ab-4f3f-935d-1b6592acfdfd None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:04.745234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763b268-7270-4524-ab31-2909babe8787 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:04.752246 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:04.780337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3831f0-9626-4578-babc-284944c6d679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:04.780950 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3831f0-9626-4578-babc-284944c6d679 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:04.781235 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3831f0-9626-4578-babc-284944c6d679 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:04.781511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3831f0-9626-4578-babc-284944c6d679 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:04.782161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3831f0-9626-4578-babc-284944c6d679 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:04.783620 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3831f0-9626-4578-babc-284944c6d679 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:04.806028 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 113/217] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 02:59:04 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:59:04.806843 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3831f0-9626-4578-babc-284944c6d679 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:04.834097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1a650-b1ab-4f3f-935d-1b6592acfdfd None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:04.874926 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e505710-6ed4-4817-96b4-06e29bc05191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:04.875836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e505710-6ed4-4817-96b4-06e29bc05191 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:04.876051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e505710-6ed4-4817-96b4-06e29bc05191 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:04.876224 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e505710-6ed4-4817-96b4-06e29bc05191 None admin] PATH_INFO: `/v3/users/dd5616c7a42e4f35b310e57d19d0c022` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:04.897174 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e505710-6ed4-4817-96b4-06e29bc05191 None admin] RBAC: Authorizing `identity:get_user(user_id=dd5616c7a42e4f35b310e57d19d0c022)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:04.898693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e505710-6ed4-4817-96b4-06e29bc05191 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:04.900526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 105/218] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:59:04 2022] GET /identity/v3/users/dd5616c7a42e4f35b310e57d19d0c022 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:04.906314 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e505710-6ed4-4817-96b4-06e29bc05191 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:04.950384 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3831f0-9626-4578-babc-284944c6d679 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:04.965373 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:04.993480 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6f73b-b29b-4db8-a284-2687689a85ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:04.994481 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6f73b-b29b-4db8-a284-2687689a85ba None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:04.994481 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6f73b-b29b-4db8-a284-2687689a85ba None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:04.994664 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6f73b-b29b-4db8-a284-2687689a85ba None admin] PATH_INFO: `/v3/users/3cb0877a442a446c806fda81718c6642` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:04.997521 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6f73b-b29b-4db8-a284-2687689a85ba None admin] RBAC: Authorizing `identity:get_user(user_id=3cb0877a442a446c806fda81718c6642)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:04.997939 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6f73b-b29b-4db8-a284-2687689a85ba None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:04.999606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 114/219] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:59:04 2022] GET /identity/v3/users/3cb0877a442a446c806fda81718c6642 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:05.005850 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6f73b-b29b-4db8-a284-2687689a85ba None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:05.021766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e505710-6ed4-4817-96b4-06e29bc05191 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:05.044581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:05.064322 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e26a3-7af3-4620-9e1c-7a88a928d565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:05.065223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e26a3-7af3-4620-9e1c-7a88a928d565 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:05.065601 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e26a3-7af3-4620-9e1c-7a88a928d565 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:05.065778 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e26a3-7af3-4620-9e1c-7a88a928d565 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:05.066511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e26a3-7af3-4620-9e1c-7a88a928d565 None admin] RBAC: Adding query filter params (role.id=11de9783bae3450ea53955de9ebcc9d1, user.id=dd5616c7a42e4f35b310e57d19d0c022, scope.system=all) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:05.067396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e26a3-7af3-4620-9e1c-7a88a928d565 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:05.068832 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e26a3-7af3-4620-9e1c-7a88a928d565 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:05.074887 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 106/220] 10.222.0.27 () {56 vars in 1308 bytes} [Mon Oct 17 02:59:04 2022] GET /identity/v3/role_assignments?user.id=dd5616c7a42e4f35b310e57d19d0c022&scope.system=all&role.id=11de9783bae3450ea53955de9ebcc9d1 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:05.130470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6f73b-b29b-4db8-a284-2687689a85ba None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:05.171339 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0b015-3c9d-4e99-8400-7f308a765953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:05.171972 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0b015-3c9d-4e99-8400-7f308a765953 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:05.172249 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0b015-3c9d-4e99-8400-7f308a765953 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:05.172522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0b015-3c9d-4e99-8400-7f308a765953 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:05.253202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0b015-3c9d-4e99-8400-7f308a765953 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:05.255368 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0b015-3c9d-4e99-8400-7f308a765953 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:05.291321 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:05.370893 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 115/221] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:59:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 02:59:05.380249 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e26a3-7af3-4620-9e1c-7a88a928d565 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:05.486706 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e26a3-7af3-4620-9e1c-7a88a928d565 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:05.503519 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:05.524476 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f053c22-3746-48bc-8308-ed6f0013f48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:05.525126 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f053c22-3746-48bc-8308-ed6f0013f48f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:05.525407 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f053c22-3746-48bc-8308-ed6f0013f48f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:05.525683 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f053c22-3746-48bc-8308-ed6f0013f48f None admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:05.526823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f053c22-3746-48bc-8308-ed6f0013f48f None admin] RBAC: Authorizing `identity:get_project(project_id=615d529fe91a4c82b4d4fe25f10a0478)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:05.528315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f053c22-3746-48bc-8308-ed6f0013f48f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:05.530130 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 107/222] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:59:05 2022] GET /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:05.536536 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0b015-3c9d-4e99-8400-7f308a765953 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:05.638726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0b015-3c9d-4e99-8400-7f308a765953 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:05.669584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:05.679168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd64602-5b96-4117-a27e-50a1de23f240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:05.679790 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd64602-5b96-4117-a27e-50a1de23f240 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:05.680072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd64602-5b96-4117-a27e-50a1de23f240 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:05.680348 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd64602-5b96-4117-a27e-50a1de23f240 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:05.681089 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd64602-5b96-4117-a27e-50a1de23f240 None admin] RBAC: Adding query filter params (role.id=1dc8aea1b26145879f699bdeb2f077da, scope.project.id=615d529fe91a4c82b4d4fe25f10a0478, user.id=3cb0877a442a446c806fda81718c6642) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:05.681407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd64602-5b96-4117-a27e-50a1de23f240 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:05.682844 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd64602-5b96-4117-a27e-50a1de23f240 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:05.689015 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 116/223] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 02:59:05 2022] GET /identity/v3/role_assignments?user.id=3cb0877a442a446c806fda81718c6642&scope.project.id=615d529fe91a4c82b4d4fe25f10a0478&role.id=1dc8aea1b26145879f699bdeb2f077da => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:07.169294 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc2b8d-9304-4741-814d-f1ddf57462bf None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:07.169804 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc2b8d-9304-4741-814d-f1ddf57462bf None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:07.169877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc2b8d-9304-4741-814d-f1ddf57462bf None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:07.171228 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 108/224] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:07.176996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54539ad1-7800-42c4-9b4d-64cc87bb115e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:07.177439 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54539ad1-7800-42c4-9b4d-64cc87bb115e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:07.177582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54539ad1-7800-42c4-9b4d-64cc87bb115e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:07.197552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-54539ad1-7800-42c4-9b4d-64cc87bb115e None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:07.198767 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-54539ad1-7800-42c4-9b4d-64cc87bb115e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:07.258445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eeb0cf-dc17-4b43-83ba-44c993f7b519 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:07.258733 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eeb0cf-dc17-4b43-83ba-44c993f7b519 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:07.259025 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eeb0cf-dc17-4b43-83ba-44c993f7b519 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:07.260963 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 109/225] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:07.266032 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cb310-da96-446c-a1e3-7e673cba39ab None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:07.266318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cb310-da96-446c-a1e3-7e673cba39ab None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:07.266956 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1cb310-da96-446c-a1e3-7e673cba39ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:07.275374 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 117/226] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:07.290747 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ff1cb310-da96-446c-a1e3-7e673cba39ab None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:07.292043 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ff1cb310-da96-446c-a1e3-7e673cba39ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:07.309946 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fc642-3640-4590-9c8b-a2b28589d96f None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:07.310436 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fc642-3640-4590-9c8b-a2b28589d96f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:07.310714 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fc642-3640-4590-9c8b-a2b28589d96f None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:07.312211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 118/227] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 02:59:07.317642 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da634cc2-3611-41a7-9723-630166409dcf None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:07.317972 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da634cc2-3611-41a7-9723-630166409dcf None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:07.318250 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da634cc2-3611-41a7-9723-630166409dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:07.336981 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-da634cc2-3611-41a7-9723-630166409dcf None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:07.337885 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-da634cc2-3611-41a7-9723-630166409dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:07.363473 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 110/228] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:07.407633 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 119/229] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:59:07.545197 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cb310-da96-446c-a1e3-7e673cba39ab None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:07.637996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da634cc2-3611-41a7-9723-630166409dcf None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:07.648215 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1cb310-da96-446c-a1e3-7e673cba39ab None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:07.666954 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:07.689667 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f8657-ca78-4457-984a-1e502d7cad0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:07.690301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f8657-ca78-4457-984a-1e502d7cad0f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:07.690578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f8657-ca78-4457-984a-1e502d7cad0f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:07.690849 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f8657-ca78-4457-984a-1e502d7cad0f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:07.698859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:07.702108 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f8657-ca78-4457-984a-1e502d7cad0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:07.706970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-6e9f8657-ca78-4457-984a-1e502d7cad0f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 02:59:07.706970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:07.706970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:07.706970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:07.706970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:07.706970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:07.706970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:07.706970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:07.706970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:07.706970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:07.706970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:07.706970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:07.706970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:07.706970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:07.706970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:07.706970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:07.707928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:07.707928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:07.707928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:07.707928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:07.707928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:07.707928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:07.707928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:07.707928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:07.707928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:07.707928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:07.707928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:07.707928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:07.707928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:07.707928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:07.707928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:07.707928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:07.708734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:07.708734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:07.708734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:07.708734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:07.708734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:07.708734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:07.708734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:07.708734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:07.708734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:07.708734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:07.708734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 02:59:07.708734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:07.708734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 111/230] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:07.709413 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f8657-ca78-4457-984a-1e502d7cad0f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:07.744647 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da634cc2-3611-41a7-9723-630166409dcf None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:07.778272 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298da089-e308-471b-bc7f-efe4f72959cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:07.778933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298da089-e308-471b-bc7f-efe4f72959cd None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:07.779225 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298da089-e308-471b-bc7f-efe4f72959cd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:07.779499 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298da089-e308-471b-bc7f-efe4f72959cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:07.780601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298da089-e308-471b-bc7f-efe4f72959cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:07.782040 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298da089-e308-471b-bc7f-efe4f72959cd None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:07.783663 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 120/231] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:07.783663 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298da089-e308-471b-bc7f-efe4f72959cd None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:07.810508 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f8657-ca78-4457-984a-1e502d7cad0f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:07.844035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e3c75-f115-4612-abdd-d5af7ec4b8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:07.844663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e3c75-f115-4612-abdd-d5af7ec4b8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:07.844949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e3c75-f115-4612-abdd-d5af7ec4b8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:07.845219 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e3c75-f115-4612-abdd-d5af7ec4b8e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:07.855235 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86e3c75-f115-4612-abdd-d5af7ec4b8e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:07.860072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-d86e3c75-f115-4612-abdd-d5af7ec4b8e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:59:07.860072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:07.860072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:07.860072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:07.860072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:07.860072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:07.860072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:07.860072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:07.860072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:07.860072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:07.860072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:07.860072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:07.860072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:07.860072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:07.860072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:07.860072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:07.861011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:07.861011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:07.861011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:07.861011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:07.861011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:07.861011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:07.861011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:07.861011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:07.861011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:07.861011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:07.861011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:07.861011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:07.861011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:07.861011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:07.861011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:07.861011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:07.861818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:07.861818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:07.861818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:07.861818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:07.861818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:07.861818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:07.861818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:07.861818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:07.861818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:07.861818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:07.861818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:59:07.861818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:07.861818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 112/232] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:07.862506 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e3c75-f115-4612-abdd-d5af7ec4b8e2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:07.890177 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298da089-e308-471b-bc7f-efe4f72959cd None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:07.900114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:07.913265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:07.925061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca2880-cd98-47e1-b3ba-15a09bf2473f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:07.925669 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca2880-cd98-47e1-b3ba-15a09bf2473f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:07.925966 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca2880-cd98-47e1-b3ba-15a09bf2473f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:07.926409 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca2880-cd98-47e1-b3ba-15a09bf2473f None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:07.927246 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca2880-cd98-47e1-b3ba-15a09bf2473f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:07.927552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca2880-cd98-47e1-b3ba-15a09bf2473f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:07.928912 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca2880-cd98-47e1-b3ba-15a09bf2473f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:07.936678 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 121/233] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:59:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:07.937537 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca2880-cd98-47e1-b3ba-15a09bf2473f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:07.967378 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e3c75-f115-4612-abdd-d5af7ec4b8e2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:08.001244 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb3462-3a0c-4a86-9c08-b00d9e18466b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:08.001850 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb3462-3a0c-4a86-9c08-b00d9e18466b None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:08.002136 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb3462-3a0c-4a86-9c08-b00d9e18466b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:08.002408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb3462-3a0c-4a86-9c08-b00d9e18466b None admin] PATH_INFO: `/v3/groups` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:08.003069 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb3462-3a0c-4a86-9c08-b00d9e18466b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:08.005424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb3462-3a0c-4a86-9c08-b00d9e18466b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:08.021034 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 113/234] 10.222.0.27 () {60 vars in 1134 bytes} [Mon Oct 17 02:59:07 2022] POST /identity/v3/groups => generated 229 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:59:08.021849 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb3462-3a0c-4a86-9c08-b00d9e18466b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:08.037800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca2880-cd98-47e1-b3ba-15a09bf2473f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:08.075916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49487bec-ebbf-4351-8e11-cfe29d9cd3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:08.076472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49487bec-ebbf-4351-8e11-cfe29d9cd3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:08.076756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49487bec-ebbf-4351-8e11-cfe29d9cd3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:08.077032 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49487bec-ebbf-4351-8e11-cfe29d9cd3c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:08.077753 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49487bec-ebbf-4351-8e11-cfe29d9cd3c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:08.078062 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49487bec-ebbf-4351-8e11-cfe29d9cd3c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:08.079535 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49487bec-ebbf-4351-8e11-cfe29d9cd3c7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:08.090162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 122/235] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:59:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:08.095777 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49487bec-ebbf-4351-8e11-cfe29d9cd3c7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:08.139476 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:08.147808 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:08.151843 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb3462-3a0c-4a86-9c08-b00d9e18466b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:08.191402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f65386-c40a-440a-91ba-b19b79c4d901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:08.193510 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f65386-c40a-440a-91ba-b19b79c4d901 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:08.194027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f65386-c40a-440a-91ba-b19b79c4d901 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:08.194466 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f65386-c40a-440a-91ba-b19b79c4d901 None admin] PATH_INFO: `/v3/users/8bf7dd05302f42449e1396e695714725` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:08.207987 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49487bec-ebbf-4351-8e11-cfe29d9cd3c7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:08.211310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f65386-c40a-440a-91ba-b19b79c4d901 None admin] RBAC: Authorizing `identity:get_user(user_id=8bf7dd05302f42449e1396e695714725)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:08.213089 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f65386-c40a-440a-91ba-b19b79c4d901 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:08.214659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 114/236] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:59:08 2022] GET /identity/v3/users/8bf7dd05302f42449e1396e695714725 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:08.220628 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f65386-c40a-440a-91ba-b19b79c4d901 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:08.248889 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689d54e-2322-428b-9e15-7a9809e057d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:08.249596 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689d54e-2322-428b-9e15-7a9809e057d5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:08.249902 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689d54e-2322-428b-9e15-7a9809e057d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:08.250165 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689d54e-2322-428b-9e15-7a9809e057d5 None admin] PATH_INFO: `/v3/users/dd5616c7a42e4f35b310e57d19d0c022` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:08.251364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689d54e-2322-428b-9e15-7a9809e057d5 None admin] RBAC: Authorizing `identity:get_user(user_id=dd5616c7a42e4f35b310e57d19d0c022)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:08.252935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689d54e-2322-428b-9e15-7a9809e057d5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:08.258965 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 123/237] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:59:08 2022] GET /identity/v3/users/dd5616c7a42e4f35b310e57d19d0c022 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:08.260456 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689d54e-2322-428b-9e15-7a9809e057d5 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:08.342617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f65386-c40a-440a-91ba-b19b79c4d901 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:08.372677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689d54e-2322-428b-9e15-7a9809e057d5 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:08.382003 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308d1af-483d-479d-b7d6-db43d0017603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:08.382615 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308d1af-483d-479d-b7d6-db43d0017603 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:08.383047 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308d1af-483d-479d-b7d6-db43d0017603 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:08.383323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308d1af-483d-479d-b7d6-db43d0017603 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:08.384058 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308d1af-483d-479d-b7d6-db43d0017603 None admin] RBAC: Adding query filter params (role.id=1dc8aea1b26145879f699bdeb2f077da, user.id=8bf7dd05302f42449e1396e695714725, scope.system=all) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:08.384377 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308d1af-483d-479d-b7d6-db43d0017603 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:08.385872 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5308d1af-483d-479d-b7d6-db43d0017603 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:08.388891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:08.391283 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 115/238] 10.222.0.27 () {56 vars in 1308 bytes} [Mon Oct 17 02:59:08 2022] GET /identity/v3/role_assignments?user.id=8bf7dd05302f42449e1396e695714725&scope.system=all&role.id=1dc8aea1b26145879f699bdeb2f077da => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:08.412535 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b23ac-a143-4933-a757-b12335000b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:08.413230 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b23ac-a143-4933-a757-b12335000b6b None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:08.413514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b23ac-a143-4933-a757-b12335000b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:08.413788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b23ac-a143-4933-a757-b12335000b6b None admin] PATH_INFO: `/v3/system/users/dd5616c7a42e4f35b310e57d19d0c022/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:08.415348 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b23ac-a143-4933-a757-b12335000b6b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=dd5616c7a42e4f35b310e57d19d0c022, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:08.416770 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b23ac-a143-4933-a757-b12335000b6b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:08.435212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 124/239] 10.222.0.27 () {58 vars in 1334 bytes} [Mon Oct 17 02:59:08 2022] PUT /identity/v3/system/users/dd5616c7a42e4f35b310e57d19d0c022/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 02:59:10.198096 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a6bd4-437d-4233-b119-6c2c730f404b None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:10.198607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a6bd4-437d-4233-b119-6c2c730f404b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:10.198607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a6bd4-437d-4233-b119-6c2c730f404b None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:10.200142 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 116/240] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:10.205109 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fa5c5-bec4-4bd9-839f-168bb7347d06 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:10.205495 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fa5c5-bec4-4bd9-839f-168bb7347d06 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:10.205653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269fa5c5-bec4-4bd9-839f-168bb7347d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:10.224914 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-269fa5c5-bec4-4bd9-839f-168bb7347d06 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:10.226123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-269fa5c5-bec4-4bd9-839f-168bb7347d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:10.298756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 125/241] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:10.539946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5cf27-ae11-4a8b-8c0a-bc72505ab8fb None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:10.540280 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5cf27-ae11-4a8b-8c0a-bc72505ab8fb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:10.540452 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5cf27-ae11-4a8b-8c0a-bc72505ab8fb None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:10.541903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 117/242] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:10.546844 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6e270-0b21-4227-a96d-6677ab71cf52 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:10.547165 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6e270-0b21-4227-a96d-6677ab71cf52 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:10.547440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6e270-0b21-4227-a96d-6677ab71cf52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:10.566221 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-5fd6e270-0b21-4227-a96d-6677ab71cf52 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:10.567617 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5fd6e270-0b21-4227-a96d-6677ab71cf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:10.568549 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5cf27-ae11-4a8b-8c0a-bc72505ab8fb None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:10.587752 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:10.640970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 126/243] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:10.640970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d347a2-9d81-4587-bdbb-2b7c0dc19e23 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:10.640970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d347a2-9d81-4587-bdbb-2b7c0dc19e23 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:10.640970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d347a2-9d81-4587-bdbb-2b7c0dc19e23 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:10.644452 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 127/244] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:10.650125 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57ff16-d28a-4128-b4e7-fb7f8ffc55be None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:10.650413 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57ff16-d28a-4128-b4e7-fb7f8ffc55be None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:10.650690 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57ff16-d28a-4128-b4e7-fb7f8ffc55be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:10.670136 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8d57ff16-d28a-4128-b4e7-fb7f8ffc55be None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:10.671354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8d57ff16-d28a-4128-b4e7-fb7f8ffc55be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:10.677194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5cf27-ae11-4a8b-8c0a-bc72505ab8fb None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:10.703643 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:10.716521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebf44f-12a7-430d-86be-da05e8fd1b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:10.717145 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebf44f-12a7-430d-86be-da05e8fd1b9d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:10.717423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebf44f-12a7-430d-86be-da05e8fd1b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:10.717693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebf44f-12a7-430d-86be-da05e8fd1b9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:10.718789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aebf44f-12a7-430d-86be-da05e8fd1b9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:10.720278 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aebf44f-12a7-430d-86be-da05e8fd1b9d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:10.721993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 118/245] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:10.727642 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebf44f-12a7-430d-86be-da05e8fd1b9d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:10.749571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 128/246] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:59:10.809883 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:10.825439 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebf44f-12a7-430d-86be-da05e8fd1b9d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:10.858572 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e7b3d-4daa-4623-844b-968118a40b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:10.859203 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e7b3d-4daa-4623-844b-968118a40b80 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:10.859486 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e7b3d-4daa-4623-844b-968118a40b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:10.859759 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e7b3d-4daa-4623-844b-968118a40b80 None admin] PATH_INFO: `/v3/groups` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:10.860390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e7b3d-4daa-4623-844b-968118a40b80 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:10.861838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e7b3d-4daa-4623-844b-968118a40b80 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:10.878066 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 119/247] 10.222.0.27 () {60 vars in 1134 bytes} [Mon Oct 17 02:59:10 2022] POST /identity/v3/groups => generated 226 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 02:59:10.915819 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57ff16-d28a-4128-b4e7-fb7f8ffc55be None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:11.023886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57ff16-d28a-4128-b4e7-fb7f8ffc55be None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:11.050256 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e7b3d-4daa-4623-844b-968118a40b80 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:11.061719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1e067-ee8a-4d61-8227-5d0035fe0367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:11.062320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1e067-ee8a-4d61-8227-5d0035fe0367 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:11.062601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1e067-ee8a-4d61-8227-5d0035fe0367 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:11.062891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1e067-ee8a-4d61-8227-5d0035fe0367 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:11.073701 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1e067-ee8a-4d61-8227-5d0035fe0367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:11.078959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-14e1e067-ee8a-4d61-8227-5d0035fe0367 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 02:59:11.078959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:11.078959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:11.078959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:11.078959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:11.078959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:11.078959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:11.078959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:11.078959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:11.078959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:11.078959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:11.078959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:11.078959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:11.078959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:11.078959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:11.078959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:11.079886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:11.079886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:11.079886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:11.079886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:11.079886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:11.079886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:11.079886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:11.079886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:11.079886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:11.079886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:11.079886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:11.079886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:11.079886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:11.079886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:11.079886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:11.079886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:11.081020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:11.081020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:11.081020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:11.081020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:11.081020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:11.081020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:11.081020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:11.081020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:11.081020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:11.081020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:11.081020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 02:59:11.081020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:11.081020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 129/248] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:11.086223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1e067-ee8a-4d61-8227-5d0035fe0367 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:11.118500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:11.184633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e7b3d-4daa-4623-844b-968118a40b80 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:11.220174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1e067-ee8a-4d61-8227-5d0035fe0367 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:11.236245 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dcf9dd-c0bf-4549-8713-de035958a44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:11.237440 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcf9dd-c0bf-4549-8713-de035958a44b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:11.237891 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcf9dd-c0bf-4549-8713-de035958a44b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:11.238305 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcf9dd-c0bf-4549-8713-de035958a44b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:11.253857 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dcf9dd-c0bf-4549-8713-de035958a44b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:11.261380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-a1dcf9dd-c0bf-4549-8713-de035958a44b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:59:11.261380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:11.261380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:11.261380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:11.261380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:11.261380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:11.261380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:11.261380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:11.261380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:11.261380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:11.261380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:11.261380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:11.261380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:11.261380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:11.261380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:11.261380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:11.262481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:11.262481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:11.262481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:11.262481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:11.262481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:11.262481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:11.262481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:11.262481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:11.262481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:11.262481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:11.262481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:11.262481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:11.262481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:11.262481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:11.262481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:11.262481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:11.263366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:11.263366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:11.263366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:11.263366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:11.263366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:11.263366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:11.263366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:11.263366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:11.263366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:11.263366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:11.263366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:59:11.263366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:11.265488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 120/249] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:59:11.270327 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09685b76-b1d5-4a97-951e-c45ce1d08dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:11.270327 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09685b76-b1d5-4a97-951e-c45ce1d08dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:11.270327 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09685b76-b1d5-4a97-951e-c45ce1d08dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:11.270327 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09685b76-b1d5-4a97-951e-c45ce1d08dd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:11.270670 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09685b76-b1d5-4a97-951e-c45ce1d08dd2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:11.270670 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09685b76-b1d5-4a97-951e-c45ce1d08dd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:11.270670 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09685b76-b1d5-4a97-951e-c45ce1d08dd2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:11.272147 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dcf9dd-c0bf-4549-8713-de035958a44b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:11.277079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:11.281459 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 130/250] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:59:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:11.289669 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09685b76-b1d5-4a97-951e-c45ce1d08dd2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:11.403971 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dcf9dd-c0bf-4549-8713-de035958a44b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:11.405205 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:11.421270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09685b76-b1d5-4a97-951e-c45ce1d08dd2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:11.451200 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1940f-e842-4dba-ac39-362f6d24b67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:11.451824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1940f-e842-4dba-ac39-362f6d24b67a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:11.452103 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1940f-e842-4dba-ac39-362f6d24b67a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:11.452516 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1940f-e842-4dba-ac39-362f6d24b67a None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:11.453341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1940f-e842-4dba-ac39-362f6d24b67a None admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:11.453649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1940f-e842-4dba-ac39-362f6d24b67a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:11.455737 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1940f-e842-4dba-ac39-362f6d24b67a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:11.460966 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d4f93-9395-4d3f-92a1-7dce32ba410c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:11.461897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d4f93-9395-4d3f-92a1-7dce32ba410c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:11.462384 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d4f93-9395-4d3f-92a1-7dce32ba410c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:11.462844 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d4f93-9395-4d3f-92a1-7dce32ba410c None admin] PATH_INFO: `/v3/users/8bf7dd05302f42449e1396e695714725` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:11.464770 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d4f93-9395-4d3f-92a1-7dce32ba410c None admin] RBAC: Authorizing `identity:get_user(user_id=8bf7dd05302f42449e1396e695714725)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:11.471480 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 121/251] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:59:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:11.471626 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d4f93-9395-4d3f-92a1-7dce32ba410c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:11.471626 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 131/252] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:59:11 2022] GET /identity/v3/users/8bf7dd05302f42449e1396e695714725 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:11.475951 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1940f-e842-4dba-ac39-362f6d24b67a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:11.480659 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d4f93-9395-4d3f-92a1-7dce32ba410c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:11.561325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:11.603811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d4f93-9395-4d3f-92a1-7dce32ba410c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:11.619915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1940f-e842-4dba-ac39-362f6d24b67a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:11.654274 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01daa685-5ba9-4114-934f-909eb0768725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:11.655055 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01daa685-5ba9-4114-934f-909eb0768725 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:11.655293 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01daa685-5ba9-4114-934f-909eb0768725 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:11.655566 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01daa685-5ba9-4114-934f-909eb0768725 None admin] PATH_INFO: `/v3/system/users/8bf7dd05302f42449e1396e695714725/roles/1dc8aea1b26145879f699bdeb2f077da` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:11.657092 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01daa685-5ba9-4114-934f-909eb0768725 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8bf7dd05302f42449e1396e695714725, role_id=1dc8aea1b26145879f699bdeb2f077da)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:11.658828 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01daa685-5ba9-4114-934f-909eb0768725 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:11.666313 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2bacf-f4c4-4400-a2cc-d8f6671f382f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:11.667680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2bacf-f4c4-4400-a2cc-d8f6671f382f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:11.667680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2bacf-f4c4-4400-a2cc-d8f6671f382f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:11.667680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2bacf-f4c4-4400-a2cc-d8f6671f382f None admin] PATH_INFO: `/v3/users/dd5616c7a42e4f35b310e57d19d0c022` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:11.669067 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2bacf-f4c4-4400-a2cc-d8f6671f382f None admin] RBAC: Authorizing `identity:get_user(user_id=dd5616c7a42e4f35b310e57d19d0c022)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:11.670305 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 132/253] 10.222.0.27 () {58 vars in 1334 bytes} [Mon Oct 17 02:59:11 2022] PUT /identity/v3/system/users/8bf7dd05302f42449e1396e695714725/roles/1dc8aea1b26145879f699bdeb2f077da => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:59:11.670719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2bacf-f4c4-4400-a2cc-d8f6671f382f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:11.674204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 122/254] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:59:11 2022] GET /identity/v3/users/dd5616c7a42e4f35b310e57d19d0c022 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:11.680094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01daa685-5ba9-4114-934f-909eb0768725 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:11.711062 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:11.804096 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01daa685-5ba9-4114-934f-909eb0768725 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:11.844196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d676e27-6a8c-41e7-a509-8acc058aab74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:11.844828 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d676e27-6a8c-41e7-a509-8acc058aab74 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:11.845112 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d676e27-6a8c-41e7-a509-8acc058aab74 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:11.845387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d676e27-6a8c-41e7-a509-8acc058aab74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:11.846191 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d676e27-6a8c-41e7-a509-8acc058aab74 None admin] RBAC: Adding query filter params (role.id=11de9783bae3450ea53955de9ebcc9d1, user.id=dd5616c7a42e4f35b310e57d19d0c022, scope.system=all) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:11.846525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d676e27-6a8c-41e7-a509-8acc058aab74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:11.848610 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d676e27-6a8c-41e7-a509-8acc058aab74 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:11.866755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 133/255] 10.222.0.27 () {56 vars in 1308 bytes} [Mon Oct 17 02:59:11 2022] GET /identity/v3/role_assignments?user.id=dd5616c7a42e4f35b310e57d19d0c022&scope.system=all&role.id=11de9783bae3450ea53955de9ebcc9d1 => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:13.547101 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5c5dd-2d74-4f9e-81d1-a0c68daae0cd None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:13.548204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5c5dd-2d74-4f9e-81d1-a0c68daae0cd None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:13.548709 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5c5dd-2d74-4f9e-81d1-a0c68daae0cd None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:13.550262 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 123/256] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:13.557375 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7c7a8-24b3-4c5b-8cb4-ce4867d047bf None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:13.558314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7c7a8-24b3-4c5b-8cb4-ce4867d047bf None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:13.558851 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7c7a8-24b3-4c5b-8cb4-ce4867d047bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:13.580162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e1f7c7a8-24b3-4c5b-8cb4-ce4867d047bf None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:13.581790 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e1f7c7a8-24b3-4c5b-8cb4-ce4867d047bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:13.649155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 134/257] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 02:59:13.913406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7cf65-c137-424f-baaf-0437b44f6d9b None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:13.914062 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7cf65-c137-424f-baaf-0437b44f6d9b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:13.914533 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7cf65-c137-424f-baaf-0437b44f6d9b None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:13.916348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 124/258] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:13.924383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2315449-6d28-4137-ab59-5f8b65ec2697 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:13.925008 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2315449-6d28-4137-ab59-5f8b65ec2697 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:13.925588 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2315449-6d28-4137-ab59-5f8b65ec2697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:13.951622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a2315449-6d28-4137-ab59-5f8b65ec2697 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:13.953198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a2315449-6d28-4137-ab59-5f8b65ec2697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:13.962506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 135/259] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 02:59:13.984200 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7cf65-c137-424f-baaf-0437b44f6d9b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:14.000121 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7cf65-c137-424f-baaf-0437b44f6d9b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:14.005495 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3e0f8-882f-4bf3-9a7f-d5e8d747977d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:14.006404 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3e0f8-882f-4bf3-9a7f-d5e8d747977d admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:14.006895 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3e0f8-882f-4bf3-9a7f-d5e8d747977d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:14.007360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3e0f8-882f-4bf3-9a7f-d5e8d747977d admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:14.026274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3e0f8-882f-4bf3-9a7f-d5e8d747977d admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:14.031485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-c8e3e0f8-882f-4bf3-9a7f-d5e8d747977d admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:59:14.031485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:14.031485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:14.031485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:14.031485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:14.031485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:14.031485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:14.031485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:14.031485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:14.031485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:14.031485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:14.031485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:14.031485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:14.031485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:14.031485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:14.031485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:14.032353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:14.032353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:14.032353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:14.032353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:14.032353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:14.032353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:14.032353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:14.032353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:14.032353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:14.032353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:14.032353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:14.032353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:14.032353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:14.032353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:14.032353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:14.032353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:14.033139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:14.033139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:14.033139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:14.033139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:14.033139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:14.033139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:14.033139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:14.033139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:14.033139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:14.033139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:14.033139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:59:14.033139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:14.035941 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 125/260] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Oct 17 02:59:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 52 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:14.041839 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2315449-6d28-4137-ab59-5f8b65ec2697 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:14.055169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2315449-6d28-4137-ab59-5f8b65ec2697 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:14.059923 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e4f33-0029-44cb-b61f-920965d4b5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:14.060767 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e4f33-0029-44cb-b61f-920965d4b5cd admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:14.061239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e4f33-0029-44cb-b61f-920965d4b5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:14.061704 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e4f33-0029-44cb-b61f-920965d4b5cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:14.062591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e4f33-0029-44cb-b61f-920965d4b5cd admin admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:14.065276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e4f33-0029-44cb-b61f-920965d4b5cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:14.067520 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-819e4f33-0029-44cb-b61f-920965d4b5cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 02:59:14.067520 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Oct 17 02:59:14.067520 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: Oct 17 02:59:14.068796 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e4f33-0029-44cb-b61f-920965d4b5cd admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:14.078360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 136/261] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Oct 17 02:59:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:14.084071 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3e0f8-882f-4bf3-9a7f-d5e8d747977d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:14.097117 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3e0f8-882f-4bf3-9a7f-d5e8d747977d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:14.102485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1f92d-6ff7-4d81-a682-cce4f25d95f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:14.103371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1f92d-6ff7-4d81-a682-cce4f25d95f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:14.103864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1f92d-6ff7-4d81-a682-cce4f25d95f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:14.104324 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1f92d-6ff7-4d81-a682-cce4f25d95f2 admin admin] PATH_INFO: `/v3/groups/68a2a5a679e94871a407cedec669a34b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:14.115465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1f92d-6ff7-4d81-a682-cce4f25d95f2 admin admin] RBAC: Authorizing `identity:get_group(group_id=68a2a5a679e94871a407cedec669a34b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:14.117532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-b9a1f92d-6ff7-4d81-a682-cce4f25d95f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 02:59:14.117532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Oct 17 02:59:14.117532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: Oct 17 02:59:14.119167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1f92d-6ff7-4d81-a682-cce4f25d95f2 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:14.121001 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 126/262] 10.222.0.27 () {56 vars in 1201 bytes} [Mon Oct 17 02:59:14 2022] GET /identity/v3/groups/68a2a5a679e94871a407cedec669a34b => generated 226 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:14.126462 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e4f33-0029-44cb-b61f-920965d4b5cd admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:14.137297 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e4f33-0029-44cb-b61f-920965d4b5cd admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:14.147187 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e26316-0534-46f4-98a7-129072f54f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:14.148132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e26316-0534-46f4-98a7-129072f54f71 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:14.148619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e26316-0534-46f4-98a7-129072f54f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:14.149085 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e26316-0534-46f4-98a7-129072f54f71 admin admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:14.150689 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e26316-0534-46f4-98a7-129072f54f71 admin admin] RBAC: Authorizing `identity:get_project(project_id=f64aa9442bc9483987f3c997119762af)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:14.152973 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e26316-0534-46f4-98a7-129072f54f71 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:14.154794 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 137/263] 10.222.0.27 () {56 vars in 1207 bytes} [Mon Oct 17 02:59:14 2022] GET /identity/v3/projects/f64aa9442bc9483987f3c997119762af => generated 298 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:14.160596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1f92d-6ff7-4d81-a682-cce4f25d95f2 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:14.176540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1f92d-6ff7-4d81-a682-cce4f25d95f2 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:14.181142 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadedf7c-687c-49bc-b462-14616a585369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:14.181982 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadedf7c-687c-49bc-b462-14616a585369 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:14.182401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadedf7c-687c-49bc-b462-14616a585369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:14.182810 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadedf7c-687c-49bc-b462-14616a585369 admin admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af/groups/68a2a5a679e94871a407cedec669a34b/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:14.184853 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadedf7c-687c-49bc-b462-14616a585369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64aa9442bc9483987f3c997119762af, group_id=68a2a5a679e94871a407cedec669a34b, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:14.186020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-cadedf7c-687c-49bc-b462-14616a585369 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 02:59:14.186020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Oct 17 02:59:14.186020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: Oct 17 02:59:14.188302 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadedf7c-687c-49bc-b462-14616a585369 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:14.217162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 127/264] 10.222.0.27 () {58 vars in 1463 bytes} [Mon Oct 17 02:59:14 2022] PUT /identity/v3/projects/f64aa9442bc9483987f3c997119762af/groups/68a2a5a679e94871a407cedec669a34b/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:59:14.573238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00004b4f-50f0-47bd-9ece-bcef1ec403c7 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:14.574215 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00004b4f-50f0-47bd-9ece-bcef1ec403c7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:14.574692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00004b4f-50f0-47bd-9ece-bcef1ec403c7 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:14.576726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 138/265] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:14.581934 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d8a29-5c3e-4e68-a7fe-cfa893ae7ac1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:14.582523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d8a29-5c3e-4e68-a7fe-cfa893ae7ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:14.582744 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d8a29-5c3e-4e68-a7fe-cfa893ae7ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:14.612816 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-732d8a29-5c3e-4e68-a7fe-cfa893ae7ac1 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:14.614145 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-732d8a29-5c3e-4e68-a7fe-cfa893ae7ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:14.638982 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00004b4f-50f0-47bd-9ece-bcef1ec403c7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:14.654160 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:14.727720 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 128/266] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 02:59:14.764493 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00004b4f-50f0-47bd-9ece-bcef1ec403c7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:14.769061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a3a87-64d4-4203-9cd3-6d63ef898626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:14.769733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a3a87-64d4-4203-9cd3-6d63ef898626 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:14.770047 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a3a87-64d4-4203-9cd3-6d63ef898626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:14.770328 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a3a87-64d4-4203-9cd3-6d63ef898626 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:14.790238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a3a87-64d4-4203-9cd3-6d63ef898626 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:14.795811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-a87a3a87-64d4-4203-9cd3-6d63ef898626 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:59:14.795811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:14.795811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:14.795811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:14.795811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:14.795811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:14.795811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:14.795811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:14.795811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:14.795811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:14.795811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:14.795811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:14.795811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:14.795811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:14.795811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:14.795811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:14.796776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:14.796776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:14.796776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:14.796776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:14.796776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:14.796776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:14.796776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:14.796776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:14.796776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:14.796776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:14.796776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:14.796776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:14.796776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:14.796776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:14.796776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:14.796776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:14.797560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:14.797560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:14.797560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:14.797560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:14.797560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:14.797560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:14.797560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:14.797560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:14.797560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:14.797560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:14.797560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:59:14.797560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:14.797560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 139/267] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Oct 17 02:59:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:14.804523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d8a29-5c3e-4e68-a7fe-cfa893ae7ac1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:14.820195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d8a29-5c3e-4e68-a7fe-cfa893ae7ac1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:14.835782 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ab86c-738b-4ebd-81d1-3ba75d4e6f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:14.836347 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ab86c-738b-4ebd-81d1-3ba75d4e6f2b admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:14.836627 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ab86c-738b-4ebd-81d1-3ba75d4e6f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:14.836899 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ab86c-738b-4ebd-81d1-3ba75d4e6f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:14.837592 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ab86c-738b-4ebd-81d1-3ba75d4e6f2b admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:14.837915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ab86c-738b-4ebd-81d1-3ba75d4e6f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:14.838949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-b92ab86c-738b-4ebd-81d1-3ba75d4e6f2b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 02:59:14.838949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Oct 17 02:59:14.838949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: Oct 17 02:59:14.839861 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ab86c-738b-4ebd-81d1-3ba75d4e6f2b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:14.854706 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 129/268] 10.222.0.27 () {56 vars in 1132 bytes} [Mon Oct 17 02:59:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:14.860110 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a3a87-64d4-4203-9cd3-6d63ef898626 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:14.876515 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a3a87-64d4-4203-9cd3-6d63ef898626 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:14.892103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562936c-bef4-4b8f-ba94-bd73c17e4749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:14.892770 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562936c-bef4-4b8f-ba94-bd73c17e4749 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:14.893054 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562936c-bef4-4b8f-ba94-bd73c17e4749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:14.893328 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562936c-bef4-4b8f-ba94-bd73c17e4749 admin admin] PATH_INFO: `/v3/groups/68a2a5a679e94871a407cedec669a34b` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:14.899533 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562936c-bef4-4b8f-ba94-bd73c17e4749 admin admin] RBAC: Authorizing `identity:get_group(group_id=68a2a5a679e94871a407cedec669a34b)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:14.900577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-e562936c-bef4-4b8f-ba94-bd73c17e4749 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 02:59:14.900577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Oct 17 02:59:14.900577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: Oct 17 02:59:14.901822 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562936c-bef4-4b8f-ba94-bd73c17e4749 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:14.903745 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 140/269] 10.222.0.27 () {56 vars in 1201 bytes} [Mon Oct 17 02:59:14 2022] GET /identity/v3/groups/68a2a5a679e94871a407cedec669a34b => generated 226 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:14.909280 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ab86c-738b-4ebd-81d1-3ba75d4e6f2b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:14.923981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ab86c-738b-4ebd-81d1-3ba75d4e6f2b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:14.931081 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17bcb8b-1977-433e-a97c-298a8d40f682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:14.931716 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bcb8b-1977-433e-a97c-298a8d40f682 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:14.931995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bcb8b-1977-433e-a97c-298a8d40f682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:14.932267 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17bcb8b-1977-433e-a97c-298a8d40f682 admin admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:14.939600 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17bcb8b-1977-433e-a97c-298a8d40f682 admin admin] RBAC: Authorizing `identity:get_project(project_id=f64aa9442bc9483987f3c997119762af)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:14.941846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17bcb8b-1977-433e-a97c-298a8d40f682 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:14.943895 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 130/270] 10.222.0.27 () {56 vars in 1207 bytes} [Mon Oct 17 02:59:14 2022] GET /identity/v3/projects/f64aa9442bc9483987f3c997119762af => generated 298 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:14.949178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88598fa7-cca6-4670-9848-1cbb42bdc6ea None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:14.949468 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88598fa7-cca6-4670-9848-1cbb42bdc6ea None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:14.949748 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88598fa7-cca6-4670-9848-1cbb42bdc6ea None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:14.951297 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 141/271] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:14.951413 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17bcb8b-1977-433e-a97c-298a8d40f682 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:14.956458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cde8ee-4e35-4a88-8097-22195ae06c3a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:14.956760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cde8ee-4e35-4a88-8097-22195ae06c3a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:14.957028 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cde8ee-4e35-4a88-8097-22195ae06c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:14.964146 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17bcb8b-1977-433e-a97c-298a8d40f682 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:14.975239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12b62d-e003-4c1e-a7fe-189ef1d15ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:14.975911 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12b62d-e003-4c1e-a7fe-189ef1d15ce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:14.976192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12b62d-e003-4c1e-a7fe-189ef1d15ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:14.976465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12b62d-e003-4c1e-a7fe-189ef1d15ce9 admin admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af/groups/68a2a5a679e94871a407cedec669a34b/roles/baf207ce98e448f1a2e10c3310a0e33b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:14.984720 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12b62d-e003-4c1e-a7fe-189ef1d15ce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64aa9442bc9483987f3c997119762af, group_id=68a2a5a679e94871a407cedec669a34b, role_id=baf207ce98e448f1a2e10c3310a0e33b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:14.987461 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12b62d-e003-4c1e-a7fe-189ef1d15ce9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:14.989703 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-15cde8ee-4e35-4a88-8097-22195ae06c3a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:14.991089 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-15cde8ee-4e35-4a88-8097-22195ae06c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:15.016501 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 131/272] 10.222.0.27 () {58 vars in 1463 bytes} [Mon Oct 17 02:59:14 2022] PUT /identity/v3/projects/f64aa9442bc9483987f3c997119762af/groups/68a2a5a679e94871a407cedec669a34b/roles/baf207ce98e448f1a2e10c3310a0e33b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 02:59:15.093390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 142/273] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:15.209630 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12b62d-e003-4c1e-a7fe-189ef1d15ce9 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:15.249560 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:15.305167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12b62d-e003-4c1e-a7fe-189ef1d15ce9 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:15.314364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9dd60-36da-4569-90c2-7a8a96b2fad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:15.315060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9dd60-36da-4569-90c2-7a8a96b2fad4 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:15.315343 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9dd60-36da-4569-90c2-7a8a96b2fad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:15.315615 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9dd60-36da-4569-90c2-7a8a96b2fad4 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:15.336930 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b9dd60-36da-4569-90c2-7a8a96b2fad4 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:15.342360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-41b9dd60-36da-4569-90c2-7a8a96b2fad4 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:59:15.342360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:15.342360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:15.342360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:15.342360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:15.342360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:15.342360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:15.342360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:15.342360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:15.342360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:15.342360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:15.342360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:15.342360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:15.342360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:15.342360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:15.342360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:15.343301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:15.343301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:15.343301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:15.343301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:15.343301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:15.343301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:15.343301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:15.343301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:15.343301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:15.343301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:15.343301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:15.343301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:15.343301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:15.343301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:15.343301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:15.343301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:15.344150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:15.344150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:15.344150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:15.344150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:15.344150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:15.344150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:15.344150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:15.344150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:15.344150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:15.344150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:15.344150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:59:15.344150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:15.344150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 132/274] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Oct 17 02:59:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 134 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:15.350758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cde8ee-4e35-4a88-8097-22195ae06c3a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:15.363891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cde8ee-4e35-4a88-8097-22195ae06c3a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:15.368761 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a077d-3652-48c7-b5ad-c80b94c5bc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:15.369375 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a077d-3652-48c7-b5ad-c80b94c5bc53 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:15.369654 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a077d-3652-48c7-b5ad-c80b94c5bc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:15.369925 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a077d-3652-48c7-b5ad-c80b94c5bc53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:15.370594 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a077d-3652-48c7-b5ad-c80b94c5bc53 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:15.370919 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a077d-3652-48c7-b5ad-c80b94c5bc53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:15.372524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a077d-3652-48c7-b5ad-c80b94c5bc53 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:15.386940 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 143/275] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Oct 17 02:59:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:15.387932 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9dd60-36da-4569-90c2-7a8a96b2fad4 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:15.398564 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9dd60-36da-4569-90c2-7a8a96b2fad4 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:15.403141 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a2fbd-caba-4a60-8994-851889512394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:15.403772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a2fbd-caba-4a60-8994-851889512394 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:15.404054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a2fbd-caba-4a60-8994-851889512394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:15.404331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a2fbd-caba-4a60-8994-851889512394 admin admin] PATH_INFO: `/v3/groups/68a2a5a679e94871a407cedec669a34b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:15.411854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a2fbd-caba-4a60-8994-851889512394 admin admin] RBAC: Authorizing `identity:get_group(group_id=68a2a5a679e94871a407cedec669a34b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:15.413798 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a2fbd-caba-4a60-8994-851889512394 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:15.415605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 133/276] 10.222.0.27 () {56 vars in 1201 bytes} [Mon Oct 17 02:59:15 2022] GET /identity/v3/groups/68a2a5a679e94871a407cedec669a34b => generated 226 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:15.421067 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a077d-3652-48c7-b5ad-c80b94c5bc53 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:15.431528 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a077d-3652-48c7-b5ad-c80b94c5bc53 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:15.438834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb25767-29d5-4f2e-b3a2-ca53f67033bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:15.439487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb25767-29d5-4f2e-b3a2-ca53f67033bc admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:15.439768 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb25767-29d5-4f2e-b3a2-ca53f67033bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:15.440044 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb25767-29d5-4f2e-b3a2-ca53f67033bc admin admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:15.441183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb25767-29d5-4f2e-b3a2-ca53f67033bc admin admin] RBAC: Authorizing `identity:get_project(project_id=615d529fe91a4c82b4d4fe25f10a0478)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:15.444208 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb25767-29d5-4f2e-b3a2-ca53f67033bc admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:15.446801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 144/277] 10.222.0.27 () {56 vars in 1207 bytes} [Mon Oct 17 02:59:15 2022] GET /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478 => generated 302 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:15.451607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a2fbd-caba-4a60-8994-851889512394 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:15.464035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a2fbd-caba-4a60-8994-851889512394 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:15.474661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c0e5e-ab53-482f-8fa5-bcdd6c129101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:15.474661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c0e5e-ab53-482f-8fa5-bcdd6c129101 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:15.474661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c0e5e-ab53-482f-8fa5-bcdd6c129101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:15.474661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c0e5e-ab53-482f-8fa5-bcdd6c129101 admin admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478/groups/68a2a5a679e94871a407cedec669a34b/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:15.478771 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c0e5e-ab53-482f-8fa5-bcdd6c129101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=615d529fe91a4c82b4d4fe25f10a0478, group_id=68a2a5a679e94871a407cedec669a34b, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:15.479860 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c0e5e-ab53-482f-8fa5-bcdd6c129101 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:15.502605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 134/278] 10.222.0.27 () {58 vars in 1463 bytes} [Mon Oct 17 02:59:15 2022] PUT /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478/groups/68a2a5a679e94871a407cedec669a34b/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:59:15.605791 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb25767-29d5-4f2e-b3a2-ca53f67033bc admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:15.685173 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:15.738288 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb25767-29d5-4f2e-b3a2-ca53f67033bc admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:15.782076 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5249db6-8405-4474-b3ae-415c5ce3c449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:15.782730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5249db6-8405-4474-b3ae-415c5ce3c449 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:15.783043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5249db6-8405-4474-b3ae-415c5ce3c449 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:15.783314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5249db6-8405-4474-b3ae-415c5ce3c449 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:15.800814 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5249db6-8405-4474-b3ae-415c5ce3c449 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:15.807196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-b5249db6-8405-4474-b3ae-415c5ce3c449 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 02:59:15.807196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:15.807196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:15.807196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:15.807196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:15.807196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:15.807196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:15.807196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:15.807196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:15.807196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:15.807196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:15.807196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:15.807196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:15.807196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:15.807196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:15.807196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:15.808111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:15.808111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:15.808111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:15.808111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:15.808111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:15.808111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:15.808111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:15.808111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:15.808111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:15.808111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:15.808111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:15.808111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:15.808111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:15.808111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:15.808111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:15.808111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:15.808916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:15.808916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:15.808916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:15.808916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:15.808916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:15.808916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:15.808916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:15.808916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:15.808916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:15.808916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:15.808916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 02:59:15.808916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:15.812905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 145/279] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:15.814143 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c0e5e-ab53-482f-8fa5-bcdd6c129101 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:15.946443 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c0e5e-ab53-482f-8fa5-bcdd6c129101 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:15.985853 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2db3d2-ef18-427d-a398-280147969909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:15.986468 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2db3d2-ef18-427d-a398-280147969909 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:15.986747 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2db3d2-ef18-427d-a398-280147969909 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:15.987045 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2db3d2-ef18-427d-a398-280147969909 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:15.987723 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2db3d2-ef18-427d-a398-280147969909 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:15.988027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2db3d2-ef18-427d-a398-280147969909 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:15.989827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2db3d2-ef18-427d-a398-280147969909 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:15.999430 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 135/280] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:59:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:16.006141 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5249db6-8405-4474-b3ae-415c5ce3c449 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:16.136748 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5249db6-8405-4474-b3ae-415c5ce3c449 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:16.156589 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:16.182369 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f5395-a582-49ae-9dc1-ae04eb3e164a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:16.183045 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f5395-a582-49ae-9dc1-ae04eb3e164a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:16.183332 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f5395-a582-49ae-9dc1-ae04eb3e164a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:16.183607 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f5395-a582-49ae-9dc1-ae04eb3e164a None admin] PATH_INFO: `/v3/users/8bf7dd05302f42449e1396e695714725` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:16.184782 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f5395-a582-49ae-9dc1-ae04eb3e164a None admin] RBAC: Authorizing `identity:get_user(user_id=8bf7dd05302f42449e1396e695714725)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:16.186236 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f5395-a582-49ae-9dc1-ae04eb3e164a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:16.188212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 146/281] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 02:59:16 2022] GET /identity/v3/users/8bf7dd05302f42449e1396e695714725 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:16.194238 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2db3d2-ef18-427d-a398-280147969909 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:16.205968 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:16.351895 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03fec3-066e-4b47-b700-67005539feb7 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:16.352214 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03fec3-066e-4b47-b700-67005539feb7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:16.352455 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03fec3-066e-4b47-b700-67005539feb7 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:16.352937 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2db3d2-ef18-427d-a398-280147969909 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:16.354989 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 147/282] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:16.361373 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8573556-91cf-4c73-bde5-a9eeabf9ad29 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:16.361652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8573556-91cf-4c73-bde5-a9eeabf9ad29 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:16.361932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8573556-91cf-4c73-bde5-a9eeabf9ad29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:16.401672 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-b8573556-91cf-4c73-bde5-a9eeabf9ad29 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:16.402921 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b8573556-91cf-4c73-bde5-a9eeabf9ad29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:16.416980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c5ba3-bdaf-4815-a377-d736492e3fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:16.417607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c5ba3-bdaf-4815-a377-d736492e3fee None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:16.417889 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c5ba3-bdaf-4815-a377-d736492e3fee None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:16.418168 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c5ba3-bdaf-4815-a377-d736492e3fee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:16.418946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c5ba3-bdaf-4815-a377-d736492e3fee None admin] RBAC: Adding query filter params (role.id=1dc8aea1b26145879f699bdeb2f077da, user.id=8bf7dd05302f42449e1396e695714725, scope.system=all) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:16.419267 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c5ba3-bdaf-4815-a377-d736492e3fee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:16.420740 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c5ba3-bdaf-4815-a377-d736492e3fee None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:16.446301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 136/283] 10.222.0.27 () {56 vars in 1308 bytes} [Mon Oct 17 02:59:16 2022] GET /identity/v3/role_assignments?user.id=8bf7dd05302f42449e1396e695714725&scope.system=all&role.id=1dc8aea1b26145879f699bdeb2f077da => generated 510 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:16.508727 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 148/284] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 17 02:59:16.743824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22ce8d-9f5e-4847-8132-853d5004f5de None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:16.744159 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22ce8d-9f5e-4847-8132-853d5004f5de None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:16.744391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22ce8d-9f5e-4847-8132-853d5004f5de None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:16.746099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 137/285] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:16.751218 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4be57-0def-4c5e-b1f2-d270d5788d48 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:16.751666 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4be57-0def-4c5e-b1f2-d270d5788d48 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:16.751854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4be57-0def-4c5e-b1f2-d270d5788d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:16.784459 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f7f4be57-0def-4c5e-b1f2-d270d5788d48 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:16.786142 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f7f4be57-0def-4c5e-b1f2-d270d5788d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:16.800455 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 149/286] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 02:59:17.052615 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22ce8d-9f5e-4847-8132-853d5004f5de None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:17.069429 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22ce8d-9f5e-4847-8132-853d5004f5de None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:17.075657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58c089-018c-43ae-8659-5dc469f9249a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:17.076301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58c089-018c-43ae-8659-5dc469f9249a admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:17.076584 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58c089-018c-43ae-8659-5dc469f9249a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:17.076857 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58c089-018c-43ae-8659-5dc469f9249a admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:17.091915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58c089-018c-43ae-8659-5dc469f9249a admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:17.097036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-8d58c089-018c-43ae-8659-5dc469f9249a admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:59:17.097036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:17.097036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:17.097036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:17.097036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:17.097036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:17.097036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:17.097036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:17.097036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:17.097036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:17.097036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:17.097036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:17.097036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:17.097036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:17.097036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:17.097036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:17.098588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:17.098588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:17.098588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:17.098588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:17.098588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:17.098588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:17.098588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:17.098588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:17.098588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:17.098588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:17.098588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:17.098588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:17.098588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:17.098588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:17.098588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:17.098588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:17.099488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:17.099488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:17.099488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:17.099488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:17.099488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:17.099488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:17.099488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:17.099488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:17.099488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:17.099488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:17.099488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:59:17.099488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:17.099488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 138/287] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Oct 17 02:59:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:17.105332 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4be57-0def-4c5e-b1f2-d270d5788d48 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:17.120167 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4be57-0def-4c5e-b1f2-d270d5788d48 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:17.130736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201170f5-1f47-4324-beb0-a613f438befd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:17.131387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201170f5-1f47-4324-beb0-a613f438befd admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:17.131665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201170f5-1f47-4324-beb0-a613f438befd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:17.131935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201170f5-1f47-4324-beb0-a613f438befd admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:17.132613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201170f5-1f47-4324-beb0-a613f438befd admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:17.132961 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201170f5-1f47-4324-beb0-a613f438befd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:17.134520 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201170f5-1f47-4324-beb0-a613f438befd admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:17.146947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 150/288] 10.222.0.27 () {56 vars in 1132 bytes} [Mon Oct 17 02:59:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:17.152378 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58c089-018c-43ae-8659-5dc469f9249a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:17.168080 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58c089-018c-43ae-8659-5dc469f9249a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:17.179619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ce476-a814-4bd0-a456-3f2efa6a2965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:17.180273 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ce476-a814-4bd0-a456-3f2efa6a2965 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:17.180538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ce476-a814-4bd0-a456-3f2efa6a2965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:17.180813 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1ce476-a814-4bd0-a456-3f2efa6a2965 admin admin] PATH_INFO: `/v3/groups/68a2a5a679e94871a407cedec669a34b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:17.187622 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ce476-a814-4bd0-a456-3f2efa6a2965 admin admin] RBAC: Authorizing `identity:get_group(group_id=68a2a5a679e94871a407cedec669a34b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:17.189596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1ce476-a814-4bd0-a456-3f2efa6a2965 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:17.191660 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 139/289] 10.222.0.27 () {56 vars in 1201 bytes} [Mon Oct 17 02:59:17 2022] GET /identity/v3/groups/68a2a5a679e94871a407cedec669a34b => generated 226 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:17.197501 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201170f5-1f47-4324-beb0-a613f438befd admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:17.212034 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201170f5-1f47-4324-beb0-a613f438befd admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:17.218858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a4123-5c36-42a9-ab30-719d059586a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:17.219527 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a4123-5c36-42a9-ab30-719d059586a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:17.219807 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a4123-5c36-42a9-ab30-719d059586a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:17.220075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a4123-5c36-42a9-ab30-719d059586a0 admin admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:17.221201 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a4123-5c36-42a9-ab30-719d059586a0 admin admin] RBAC: Authorizing `identity:get_project(project_id=615d529fe91a4c82b4d4fe25f10a0478)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:17.223357 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a4123-5c36-42a9-ab30-719d059586a0 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:17.230568 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ce476-a814-4bd0-a456-3f2efa6a2965 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:17.231038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 151/290] 10.222.0.27 () {56 vars in 1207 bytes} [Mon Oct 17 02:59:17 2022] GET /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478 => generated 302 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:17.249193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1ce476-a814-4bd0-a456-3f2efa6a2965 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:17.259908 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111573db-5386-41b3-9e01-48c64f80edb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:17.260588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111573db-5386-41b3-9e01-48c64f80edb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:17.260874 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111573db-5386-41b3-9e01-48c64f80edb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:17.261151 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111573db-5386-41b3-9e01-48c64f80edb1 admin admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478/groups/68a2a5a679e94871a407cedec669a34b/roles/baf207ce98e448f1a2e10c3310a0e33b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:17.264687 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111573db-5386-41b3-9e01-48c64f80edb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=615d529fe91a4c82b4d4fe25f10a0478, group_id=68a2a5a679e94871a407cedec669a34b, role_id=baf207ce98e448f1a2e10c3310a0e33b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:17.267364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111573db-5386-41b3-9e01-48c64f80edb1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:17.273455 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a4123-5c36-42a9-ab30-719d059586a0 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:17.293890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a4123-5c36-42a9-ab30-719d059586a0 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:17.297800 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 140/291] 10.222.0.27 () {58 vars in 1463 bytes} [Mon Oct 17 02:59:17 2022] PUT /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478/groups/68a2a5a679e94871a407cedec669a34b/roles/baf207ce98e448f1a2e10c3310a0e33b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:59:17.372720 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214758de-bd5b-42a6-9c40-0c65383b6fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:17.373873 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214758de-bd5b-42a6-9c40-0c65383b6fdd admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:17.374371 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214758de-bd5b-42a6-9c40-0c65383b6fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:17.374880 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214758de-bd5b-42a6-9c40-0c65383b6fdd admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:17.394327 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214758de-bd5b-42a6-9c40-0c65383b6fdd admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:17.400968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-214758de-bd5b-42a6-9c40-0c65383b6fdd admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:59:17.400968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:17.400968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:17.400968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:17.400968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:17.400968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:17.400968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:17.400968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:17.400968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:17.400968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:17.400968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:17.400968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:17.400968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:17.400968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:17.400968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:17.400968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:17.401962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:17.401962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:17.401962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:17.401962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:17.401962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:17.401962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:17.401962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:17.401962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:17.401962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:17.401962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:17.401962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:17.401962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:17.401962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:17.401962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:17.401962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:17.401962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:17.402762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:17.402762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:17.402762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:17.402762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:17.402762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:17.402762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:17.402762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:17.402762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:17.402762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:17.402762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:17.402762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:59:17.402762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:17.405256 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 152/292] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Oct 17 02:59:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 133 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:59:17.411602 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111573db-5386-41b3-9e01-48c64f80edb1 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:17.433225 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111573db-5386-41b3-9e01-48c64f80edb1 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:17.437687 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523dbc9-d3f6-4ae2-8e9e-e1f3422ba91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:17.438368 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523dbc9-d3f6-4ae2-8e9e-e1f3422ba91c admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:17.438650 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523dbc9-d3f6-4ae2-8e9e-e1f3422ba91c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:17.438944 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523dbc9-d3f6-4ae2-8e9e-e1f3422ba91c admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:17.443809 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523dbc9-d3f6-4ae2-8e9e-e1f3422ba91c admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:17.444128 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523dbc9-d3f6-4ae2-8e9e-e1f3422ba91c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:17.445922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523dbc9-d3f6-4ae2-8e9e-e1f3422ba91c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:17.462457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 141/293] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Oct 17 02:59:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:17.468453 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214758de-bd5b-42a6-9c40-0c65383b6fdd admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:17.484387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214758de-bd5b-42a6-9c40-0c65383b6fdd admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:17.491839 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a873a-1f1b-492d-88a5-b4afe88ff190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:17.492874 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a873a-1f1b-492d-88a5-b4afe88ff190 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:17.493385 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a873a-1f1b-492d-88a5-b4afe88ff190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:17.493870 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8a873a-1f1b-492d-88a5-b4afe88ff190 admin admin] PATH_INFO: `/v3/groups/93f70af99a1a4c929be06c49c74f96d4` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:17.506309 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a873a-1f1b-492d-88a5-b4afe88ff190 admin admin] RBAC: Authorizing `identity:get_group(group_id=93f70af99a1a4c929be06c49c74f96d4)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:17.508715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8a873a-1f1b-492d-88a5-b4afe88ff190 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:17.510840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 153/294] 10.222.0.27 () {56 vars in 1201 bytes} [Mon Oct 17 02:59:17 2022] GET /identity/v3/groups/93f70af99a1a4c929be06c49c74f96d4 => generated 229 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:17.516630 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523dbc9-d3f6-4ae2-8e9e-e1f3422ba91c admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:17.531084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523dbc9-d3f6-4ae2-8e9e-e1f3422ba91c admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:17.536329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c8270-1ec6-4d10-abf5-3fe73e8b6883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:17.537040 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c8270-1ec6-4d10-abf5-3fe73e8b6883 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:17.537318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c8270-1ec6-4d10-abf5-3fe73e8b6883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:17.537590 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c8270-1ec6-4d10-abf5-3fe73e8b6883 admin admin] PATH_INFO: `/v3/projects/eb6be797d9e4443fad02ceee69bd83a7` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:17.538845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c8270-1ec6-4d10-abf5-3fe73e8b6883 admin admin] RBAC: Authorizing `identity:get_project(project_id=eb6be797d9e4443fad02ceee69bd83a7)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:17.540847 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c8270-1ec6-4d10-abf5-3fe73e8b6883 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:17.547019 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 142/295] 10.222.0.27 () {56 vars in 1207 bytes} [Mon Oct 17 02:59:17 2022] GET /identity/v3/projects/eb6be797d9e4443fad02ceee69bd83a7 => generated 344 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:17.548563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a873a-1f1b-492d-88a5-b4afe88ff190 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:17.563393 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8a873a-1f1b-492d-88a5-b4afe88ff190 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:17.571644 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c8ce0-0c2d-48cb-851c-69843dd04e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:17.572708 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c8ce0-0c2d-48cb-851c-69843dd04e19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:17.573197 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c8ce0-0c2d-48cb-851c-69843dd04e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:17.573701 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c8ce0-0c2d-48cb-851c-69843dd04e19 admin admin] PATH_INFO: `/v3/projects/eb6be797d9e4443fad02ceee69bd83a7/groups/93f70af99a1a4c929be06c49c74f96d4/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:17.578269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c8ce0-0c2d-48cb-851c-69843dd04e19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb6be797d9e4443fad02ceee69bd83a7, group_id=93f70af99a1a4c929be06c49c74f96d4, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:17.579739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-fe4c8ce0-0c2d-48cb-851c-69843dd04e19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 02:59:17.579739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Oct 17 02:59:17.579739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: Oct 17 02:59:17.582117 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c8ce0-0c2d-48cb-851c-69843dd04e19 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:17.600281 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 154/296] 10.222.0.27 () {58 vars in 1463 bytes} [Mon Oct 17 02:59:17 2022] PUT /identity/v3/projects/eb6be797d9e4443fad02ceee69bd83a7/groups/93f70af99a1a4c929be06c49c74f96d4/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:59:17.697868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b005a6c-31c7-4631-8486-d7b7c06b1a76 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:17.698049 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b005a6c-31c7-4631-8486-d7b7c06b1a76 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:17.698353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b005a6c-31c7-4631-8486-d7b7c06b1a76 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:17.700019 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 143/297] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:17.705785 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f95ad-63f6-4fd0-a1f2-14f93b92dd48 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:17.706417 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f95ad-63f6-4fd0-a1f2-14f93b92dd48 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:17.706910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f95ad-63f6-4fd0-a1f2-14f93b92dd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:17.727999 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-313f95ad-63f6-4fd0-a1f2-14f93b92dd48 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:17.729581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-313f95ad-63f6-4fd0-a1f2-14f93b92dd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:17.764445 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:17.817246 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 155/298] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:18.117781 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b005a6c-31c7-4631-8486-d7b7c06b1a76 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:18.144067 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:18.210911 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4816ccc-9481-4f7c-a5a3-cb8158f5d2b4 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:18.210911 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4816ccc-9481-4f7c-a5a3-cb8158f5d2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:18.211632 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4816ccc-9481-4f7c-a5a3-cb8158f5d2b4 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:18.220277 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 156/299] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:18 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:18.228262 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f61129-c5d2-4e7f-8e25-b7947987cfee None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:18.228819 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f61129-c5d2-4e7f-8e25-b7947987cfee None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:18.229294 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f61129-c5d2-4e7f-8e25-b7947987cfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:18.248243 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b005a6c-31c7-4631-8486-d7b7c06b1a76 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:18.260104 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d8f61129-c5d2-4e7f-8e25-b7947987cfee None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:18.261671 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d8f61129-c5d2-4e7f-8e25-b7947987cfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:18.286228 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d525f1-8f2d-4360-a994-2c7501783122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:18.290925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d525f1-8f2d-4360-a994-2c7501783122 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:18.290925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d525f1-8f2d-4360-a994-2c7501783122 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:18.290925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d525f1-8f2d-4360-a994-2c7501783122 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:18.298730 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d525f1-8f2d-4360-a994-2c7501783122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:18.305806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-44d525f1-8f2d-4360-a994-2c7501783122 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:59:18.305806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:18.305806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:18.305806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:18.305806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:18.305806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:18.305806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:18.305806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:18.305806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:18.305806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:18.305806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:18.305806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:18.305806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:18.305806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:18.305806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:18.305806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:18.306722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:18.306722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:18.306722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:18.306722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:18.306722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:18.306722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:18.306722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:18.306722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:18.306722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:18.306722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:18.306722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:18.306722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:18.306722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:18.306722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:18.306722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:18.306722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:18.307578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:18.307578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:18.307578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:18.307578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:18.307578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:18.307578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:18.307578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:18.307578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:18.307578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:18.307578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:18.307578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:59:18.307578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:18.307578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 144/300] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:18.315619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d525f1-8f2d-4360-a994-2c7501783122 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:18.401573 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 157/301] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:18.433149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d525f1-8f2d-4360-a994-2c7501783122 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:18.436070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:18.469993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d213ee0-df93-443f-9189-86ac2baa6b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:18.470604 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d213ee0-df93-443f-9189-86ac2baa6b28 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:18.470893 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d213ee0-df93-443f-9189-86ac2baa6b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:18.471173 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d213ee0-df93-443f-9189-86ac2baa6b28 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:18.471842 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d213ee0-df93-443f-9189-86ac2baa6b28 None admin] RBAC: Adding query filter params (name=member) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:18.472140 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d213ee0-df93-443f-9189-86ac2baa6b28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:18.473658 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d213ee0-df93-443f-9189-86ac2baa6b28 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:18.482592 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 145/302] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:59:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:18.488402 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f61129-c5d2-4e7f-8e25-b7947987cfee None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:18.587306 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:18.599938 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f61129-c5d2-4e7f-8e25-b7947987cfee None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:18.636659 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3125e-04a9-4a46-8a9d-5b6116c63a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:18.637271 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3125e-04a9-4a46-8a9d-5b6116c63a0a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:18.637586 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3125e-04a9-4a46-8a9d-5b6116c63a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:18.637861 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3125e-04a9-4a46-8a9d-5b6116c63a0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:18.732190 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3125e-04a9-4a46-8a9d-5b6116c63a0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:18.733639 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3125e-04a9-4a46-8a9d-5b6116c63a0a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:18.818211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ab65c-5bd1-4bc9-881c-b59c14d25d84 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:18.818517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ab65c-5bd1-4bc9-881c-b59c14d25d84 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:18.818766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ab65c-5bd1-4bc9-881c-b59c14d25d84 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:18.820383 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 146/303] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:18.838231 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1d10e-7609-4b9e-ac50-57ec64d83d7b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:18.838512 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1d10e-7609-4b9e-ac50-57ec64d83d7b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:18.838787 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1d10e-7609-4b9e-ac50-57ec64d83d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:18.847339 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:18.859281 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 158/304] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:59:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 02:59:18.860206 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3125e-04a9-4a46-8a9d-5b6116c63a0a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:18.873553 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-dab1d10e-7609-4b9e-ac50-57ec64d83d7b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:18.873553 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-dab1d10e-7609-4b9e-ac50-57ec64d83d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:18.975332 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 147/305] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:18.977213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1d10e-7609-4b9e-ac50-57ec64d83d7b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:19.026662 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3125e-04a9-4a46-8a9d-5b6116c63a0a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:19.102891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628da8a-64cc-4461-86c9-f27bb3395ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:19.103542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628da8a-64cc-4461-86c9-f27bb3395ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:19.103820 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628da8a-64cc-4461-86c9-f27bb3395ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:19.104091 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628da8a-64cc-4461-86c9-f27bb3395ea3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:19.131849 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628da8a-64cc-4461-86c9-f27bb3395ea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:19.136684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-6628da8a-64cc-4461-86c9-f27bb3395ea3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:59:19.136684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:19.136684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:19.136684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:19.136684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:19.136684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:19.136684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:19.136684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:19.136684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:19.136684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:19.136684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:19.136684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:19.136684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:19.136684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:19.136684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:19.136684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:19.137582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:19.137582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:19.137582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:19.137582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:19.137582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:19.137582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:19.137582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:19.137582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:19.137582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:19.137582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:19.137582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:19.137582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:19.137582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:19.137582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:19.137582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:19.137582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:19.138416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:19.138416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:19.138416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:19.138416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:19.138416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:19.138416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:19.138416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:19.138416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:19.138416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:19.138416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:19.138416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:59:19.138416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:19.150088 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 159/306] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Oct 17 02:59:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:19.152607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:19.168821 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628da8a-64cc-4461-86c9-f27bb3395ea3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:19.197173 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1d10e-7609-4b9e-ac50-57ec64d83d7b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:19.272428 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:19.275117 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aaccef-7f6e-43d6-9cfe-27c1fadee4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:19.275781 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaccef-7f6e-43d6-9cfe-27c1fadee4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:19.276061 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaccef-7f6e-43d6-9cfe-27c1fadee4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:19.276337 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaccef-7f6e-43d6-9cfe-27c1fadee4b3 None admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:19.277779 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aaccef-7f6e-43d6-9cfe-27c1fadee4b3 None admin] RBAC: Authorizing `identity:get_project(project_id=f64aa9442bc9483987f3c997119762af)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:19.279196 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aaccef-7f6e-43d6-9cfe-27c1fadee4b3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:19.281014 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 148/307] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:59:18 2022] GET /identity/v3/projects/f64aa9442bc9483987f3c997119762af => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:19.287401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aaccef-7f6e-43d6-9cfe-27c1fadee4b3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:19.362902 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628da8a-64cc-4461-86c9-f27bb3395ea3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:19.405796 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cd944-cd23-4650-a161-0ac47bb1076d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:19.406751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cd944-cd23-4650-a161-0ac47bb1076d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:19.407305 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cd944-cd23-4650-a161-0ac47bb1076d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:19.407803 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cd944-cd23-4650-a161-0ac47bb1076d None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:19.408714 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10cd944-cd23-4650-a161-0ac47bb1076d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:19.409220 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10cd944-cd23-4650-a161-0ac47bb1076d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:19.411082 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aaccef-7f6e-43d6-9cfe-27c1fadee4b3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:19.411631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10cd944-cd23-4650-a161-0ac47bb1076d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:19.422345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 160/308] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:19.423498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cd944-cd23-4650-a161-0ac47bb1076d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:19.458393 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ed8e1-be1e-4003-be35-3242f0b39dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:19.459426 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ed8e1-be1e-4003-be35-3242f0b39dee None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:19.459716 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ed8e1-be1e-4003-be35-3242f0b39dee None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:19.461224 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ed8e1-be1e-4003-be35-3242f0b39dee None admin] PATH_INFO: `/v3/groups/68a2a5a679e94871a407cedec669a34b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:19.462438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ed8e1-be1e-4003-be35-3242f0b39dee None admin] RBAC: Authorizing `identity:get_group(group_id=68a2a5a679e94871a407cedec669a34b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:19.464321 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ed8e1-be1e-4003-be35-3242f0b39dee None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:19.467151 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 149/309] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Oct 17 02:59:19 2022] GET /identity/v3/groups/68a2a5a679e94871a407cedec669a34b => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:19.468372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ed8e1-be1e-4003-be35-3242f0b39dee None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:19.487983 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:19.566499 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cd944-cd23-4650-a161-0ac47bb1076d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:19.584276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:19.601664 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ed8e1-be1e-4003-be35-3242f0b39dee None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:19.615442 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c140094-38ea-4102-b3cd-e5d1c9b32f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:19.616085 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c140094-38ea-4102-b3cd-e5d1c9b32f28 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:19.616360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c140094-38ea-4102-b3cd-e5d1c9b32f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:19.616628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c140094-38ea-4102-b3cd-e5d1c9b32f28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:19.634514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c140094-38ea-4102-b3cd-e5d1c9b32f28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:19.640379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-5c140094-38ea-4102-b3cd-e5d1c9b32f28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:59:19.640379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:19.640379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:19.640379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:19.640379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:19.640379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:19.640379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:19.640379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:19.640379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:19.640379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:19.640379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:19.640379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:19.640379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:19.640379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:19.640379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:19.640379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:19.641265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:19.641265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:19.641265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:19.641265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:19.641265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:19.641265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:19.641265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:19.641265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:19.641265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:19.641265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:19.641265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:19.641265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:19.641265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:19.641265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:19.641265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:19.641265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:19.642068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:19.642068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:19.642068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:19.642068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:19.642068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:19.642068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:19.642068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:19.642068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:19.642068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:19.642068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:19.642068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 02:59:19.642068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:19.642068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 161/310] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:19.644182 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c140094-38ea-4102-b3cd-e5d1c9b32f28 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:19.652397 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e305cce-4369-43b2-a9b2-bb37bde04f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:19.653032 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e305cce-4369-43b2-a9b2-bb37bde04f29 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:19.653305 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e305cce-4369-43b2-a9b2-bb37bde04f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:19.653576 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e305cce-4369-43b2-a9b2-bb37bde04f29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:19.748515 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e305cce-4369-43b2-a9b2-bb37bde04f29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:19.750660 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e305cce-4369-43b2-a9b2-bb37bde04f29 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:19.769197 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c140094-38ea-4102-b3cd-e5d1c9b32f28 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:19.769989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:19.809706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa2a9f-d121-4045-b7e1-cae3d3223f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:19.810538 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa2a9f-d121-4045-b7e1-cae3d3223f35 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:19.810960 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa2a9f-d121-4045-b7e1-cae3d3223f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:19.811339 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa2a9f-d121-4045-b7e1-cae3d3223f35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:19.812286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa2a9f-d121-4045-b7e1-cae3d3223f35 None admin] RBAC: Adding query filter params (group.id=68a2a5a679e94871a407cedec669a34b, role.id=11de9783bae3450ea53955de9ebcc9d1, scope.project.id=f64aa9442bc9483987f3c997119762af) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:19.813255 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa2a9f-d121-4045-b7e1-cae3d3223f35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:19.814903 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa2a9f-d121-4045-b7e1-cae3d3223f35 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:19.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 162/311] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Oct 17 02:59:19 2022] GET /identity/v3/role_assignments?group.id=68a2a5a679e94871a407cedec669a34b&scope.project.id=f64aa9442bc9483987f3c997119762af&role.id=11de9783bae3450ea53955de9ebcc9d1 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:19.822639 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa2a9f-d121-4045-b7e1-cae3d3223f35 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:19.887378 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:19.890205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 150/312] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:59:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 02:59:19.897564 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e305cce-4369-43b2-a9b2-bb37bde04f29 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:19.975307 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa2a9f-d121-4045-b7e1-cae3d3223f35 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:20.019688 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e305cce-4369-43b2-a9b2-bb37bde04f29 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:20.032387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb5191-c4f6-4af2-add5-2af812779e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:20.033006 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb5191-c4f6-4af2-add5-2af812779e77 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:20.033280 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb5191-c4f6-4af2-add5-2af812779e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:20.033549 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb5191-c4f6-4af2-add5-2af812779e77 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:20.034214 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb5191-c4f6-4af2-add5-2af812779e77 None admin] RBAC: Adding query filter params (name=member) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:20.034515 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb5191-c4f6-4af2-add5-2af812779e77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:20.036009 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb5191-c4f6-4af2-add5-2af812779e77 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:20.042572 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:20.047660 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 163/313] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 02:59:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:20.054328 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb5191-c4f6-4af2-add5-2af812779e77 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:20.072089 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe0054-8d0c-42c9-bbe7-9072ecb1eb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:20.072718 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe0054-8d0c-42c9-bbe7-9072ecb1eb7a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:20.073074 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe0054-8d0c-42c9-bbe7-9072ecb1eb7a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:20.073422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe0054-8d0c-42c9-bbe7-9072ecb1eb7a None admin] PATH_INFO: `/v3/projects/f64aa9442bc9483987f3c997119762af` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:20.074754 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe0054-8d0c-42c9-bbe7-9072ecb1eb7a None admin] RBAC: Authorizing `identity:get_project(project_id=f64aa9442bc9483987f3c997119762af)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:20.076260 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe0054-8d0c-42c9-bbe7-9072ecb1eb7a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:20.078229 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 151/314] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:59:19 2022] GET /identity/v3/projects/f64aa9442bc9483987f3c997119762af => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:20.083852 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe0054-8d0c-42c9-bbe7-9072ecb1eb7a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:20.165393 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:20.169006 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb5191-c4f6-4af2-add5-2af812779e77 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:20.196903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe0054-8d0c-42c9-bbe7-9072ecb1eb7a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:20.206305 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec4541-8609-41fe-8d01-bc5f0b2c540d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:20.207399 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec4541-8609-41fe-8d01-bc5f0b2c540d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:20.207883 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec4541-8609-41fe-8d01-bc5f0b2c540d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:20.208429 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec4541-8609-41fe-8d01-bc5f0b2c540d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:20.236144 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78075e-4dd6-47e3-8eec-c84b4e7dc13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:20.236877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78075e-4dd6-47e3-8eec-c84b4e7dc13c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:20.237161 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78075e-4dd6-47e3-8eec-c84b4e7dc13c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:20.237438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78075e-4dd6-47e3-8eec-c84b4e7dc13c None admin] PATH_INFO: `/v3/groups/68a2a5a679e94871a407cedec669a34b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:20.239272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78075e-4dd6-47e3-8eec-c84b4e7dc13c None admin] RBAC: Authorizing `identity:get_group(group_id=68a2a5a679e94871a407cedec669a34b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:20.240760 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78075e-4dd6-47e3-8eec-c84b4e7dc13c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:20.243248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 152/315] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Oct 17 02:59:20 2022] GET /identity/v3/groups/68a2a5a679e94871a407cedec669a34b => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:20.249182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78075e-4dd6-47e3-8eec-c84b4e7dc13c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:20.291755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec4541-8609-41fe-8d01-bc5f0b2c540d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:20.293789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec4541-8609-41fe-8d01-bc5f0b2c540d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:20.305285 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:20.362415 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78075e-4dd6-47e3-8eec-c84b4e7dc13c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:20.397663 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:20.400352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572faa8-665f-465b-a626-fa2b9bf5fbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:20.400966 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572faa8-665f-465b-a626-fa2b9bf5fbc8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:20.401330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572faa8-665f-465b-a626-fa2b9bf5fbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:20.401610 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572faa8-665f-465b-a626-fa2b9bf5fbc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:20.402448 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572faa8-665f-465b-a626-fa2b9bf5fbc8 None admin] RBAC: Adding query filter params (group.id=68a2a5a679e94871a407cedec669a34b, role.id=baf207ce98e448f1a2e10c3310a0e33b, scope.project.id=f64aa9442bc9483987f3c997119762af) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:20.402729 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572faa8-665f-465b-a626-fa2b9bf5fbc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:20.404733 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572faa8-665f-465b-a626-fa2b9bf5fbc8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:20.409835 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 164/316] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:59:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 02:59:20.412580 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390a84e-1436-4f64-86ec-c17c1480b409 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:20.412695 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 153/317] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Oct 17 02:59:20 2022] GET /identity/v3/role_assignments?group.id=68a2a5a679e94871a407cedec669a34b&scope.project.id=f64aa9442bc9483987f3c997119762af&role.id=baf207ce98e448f1a2e10c3310a0e33b => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:20.413169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390a84e-1436-4f64-86ec-c17c1480b409 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:20.413599 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572faa8-665f-465b-a626-fa2b9bf5fbc8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:20.413857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390a84e-1436-4f64-86ec-c17c1480b409 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:20.415400 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 165/318] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:20.420949 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d30fc8-4e2e-4d5a-bd55-8fd50146e4ab None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:20.421498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d30fc8-4e2e-4d5a-bd55-8fd50146e4ab None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:20.421962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d30fc8-4e2e-4d5a-bd55-8fd50146e4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:20.443700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-08d30fc8-4e2e-4d5a-bd55-8fd50146e4ab None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:20.445403 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-08d30fc8-4e2e-4d5a-bd55-8fd50146e4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:20.529524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 166/319] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:20.549125 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572faa8-665f-465b-a626-fa2b9bf5fbc8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:20.564926 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:20.593015 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21783c36-7b22-4231-aac8-fd809281f346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:20.594238 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21783c36-7b22-4231-aac8-fd809281f346 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:20.594725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21783c36-7b22-4231-aac8-fd809281f346 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:20.595217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21783c36-7b22-4231-aac8-fd809281f346 None admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:20.597397 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21783c36-7b22-4231-aac8-fd809281f346 None admin] RBAC: Authorizing `identity:get_project(project_id=615d529fe91a4c82b4d4fe25f10a0478)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:20.599904 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21783c36-7b22-4231-aac8-fd809281f346 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:20.601943 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 154/320] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:59:20 2022] GET /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:20.609196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d30fc8-4e2e-4d5a-bd55-8fd50146e4ab None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:20.656871 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23ef2-992e-4e0b-8d92-c1e1c1fcc3ab None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:20.657471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23ef2-992e-4e0b-8d92-c1e1c1fcc3ab None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:20.657942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23ef2-992e-4e0b-8d92-c1e1c1fcc3ab None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:20.659526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 155/321] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:20.666045 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446649d7-c217-44fb-afa5-b1959744ce23 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:20.666741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446649d7-c217-44fb-afa5-b1959744ce23 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:20.667052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446649d7-c217-44fb-afa5-b1959744ce23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:20.696139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-446649d7-c217-44fb-afa5-b1959744ce23 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:20.697354 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-446649d7-c217-44fb-afa5-b1959744ce23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:20.736328 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d30fc8-4e2e-4d5a-bd55-8fd50146e4ab None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:20.777221 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 156/322] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:59:20.777793 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:20.781826 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4e114-d4e6-41e5-9d5c-d24442978ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:20.782582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4e114-d4e6-41e5-9d5c-d24442978ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:20.782965 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4e114-d4e6-41e5-9d5c-d24442978ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:20.783342 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4e114-d4e6-41e5-9d5c-d24442978ad1 None admin] PATH_INFO: `/v3/groups/68a2a5a679e94871a407cedec669a34b` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:20.784937 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f4e114-d4e6-41e5-9d5c-d24442978ad1 None admin] RBAC: Authorizing `identity:get_group(group_id=68a2a5a679e94871a407cedec669a34b)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:20.788426 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f4e114-d4e6-41e5-9d5c-d24442978ad1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:20.790413 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 167/323] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Oct 17 02:59:20 2022] GET /identity/v3/groups/68a2a5a679e94871a407cedec669a34b => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:20.796136 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446649d7-c217-44fb-afa5-b1959744ce23 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:20.828616 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4e114-d4e6-41e5-9d5c-d24442978ad1 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:20.924462 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446649d7-c217-44fb-afa5-b1959744ce23 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:20.938802 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:20.954219 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4e114-d4e6-41e5-9d5c-d24442978ad1 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:20.980427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d16d3-daf6-4053-9afd-d4941df63f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:20.980960 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d16d3-daf6-4053-9afd-d4941df63f24 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:20.981238 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d16d3-daf6-4053-9afd-d4941df63f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:20.981508 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d16d3-daf6-4053-9afd-d4941df63f24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:20.982247 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d16d3-daf6-4053-9afd-d4941df63f24 None admin] RBAC: Adding query filter params (group.id=68a2a5a679e94871a407cedec669a34b, role.id=11de9783bae3450ea53955de9ebcc9d1, scope.project.id=615d529fe91a4c82b4d4fe25f10a0478) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:20.982557 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d16d3-daf6-4053-9afd-d4941df63f24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:20.984094 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d16d3-daf6-4053-9afd-d4941df63f24 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:20.991444 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 157/324] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Oct 17 02:59:20 2022] GET /identity/v3/role_assignments?group.id=68a2a5a679e94871a407cedec669a34b&scope.project.id=615d529fe91a4c82b4d4fe25f10a0478&role.id=11de9783bae3450ea53955de9ebcc9d1 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:21.009755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821cdf25-0747-433f-8219-9eab6b34e3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:21.010712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cdf25-0747-433f-8219-9eab6b34e3da None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:21.011183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cdf25-0747-433f-8219-9eab6b34e3da None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:21.011605 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821cdf25-0747-433f-8219-9eab6b34e3da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:21.027826 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821cdf25-0747-433f-8219-9eab6b34e3da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:21.033038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-821cdf25-0747-433f-8219-9eab6b34e3da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:59:21.033038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:21.033038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:21.033038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:21.033038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:21.033038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:21.033038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:21.033038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:21.033038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:21.033038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:21.033038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:21.033038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:21.033038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:21.033038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:21.033038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:21.033038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:21.034140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:21.034140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:21.034140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:21.034140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:21.034140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:21.034140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:21.034140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:21.034140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:21.034140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:21.034140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:21.034140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:21.034140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:21.034140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:21.034140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:21.034140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:21.034140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:21.035041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:21.035041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:21.035041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:21.035041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:21.035041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:21.035041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:21.035041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:21.035041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:21.035041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:21.035041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:21.035041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:59:21.035041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:21.035041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 168/325] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Oct 17 02:59:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:21.041318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d16d3-daf6-4053-9afd-d4941df63f24 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:21.056291 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821cdf25-0747-433f-8219-9eab6b34e3da None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:21.135140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:21.188830 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d16d3-daf6-4053-9afd-d4941df63f24 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:21.189653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821cdf25-0747-433f-8219-9eab6b34e3da None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:21.236401 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62d631-d0da-47cd-9156-fce9b2b5d73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:21.236826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798d6f1-1236-4340-9f31-2c12747d1ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:21.237269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62d631-d0da-47cd-9156-fce9b2b5d73c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:21.237358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62d631-d0da-47cd-9156-fce9b2b5d73c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:21.237428 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798d6f1-1236-4340-9f31-2c12747d1ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:21.237641 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62d631-d0da-47cd-9156-fce9b2b5d73c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:21.237737 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798d6f1-1236-4340-9f31-2c12747d1ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:21.237949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798d6f1-1236-4340-9f31-2c12747d1ba4 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:21.239956 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798d6f1-1236-4340-9f31-2c12747d1ba4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:21.240264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798d6f1-1236-4340-9f31-2c12747d1ba4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:21.241650 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798d6f1-1236-4340-9f31-2c12747d1ba4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:21.251251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 158/326] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:21.252280 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62d631-d0da-47cd-9156-fce9b2b5d73c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:21.257620 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798d6f1-1236-4340-9f31-2c12747d1ba4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:21.257769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-2b62d631-d0da-47cd-9156-fce9b2b5d73c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:59:21.257769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:21.257769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:21.257769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:21.257769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:21.257769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:21.257769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:21.257769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:21.257769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:21.257769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:21.257769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:21.257769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:21.257769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:21.257769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:21.257769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:21.257769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:21.258601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:21.258601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:21.258601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:21.258601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:21.258601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:21.258601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:21.258601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:21.258601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:21.258601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:21.258601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:21.258601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:21.258601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:21.258601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:21.258601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:21.258601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:21.258601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:21.259440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:21.259440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:21.259440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:21.259440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:21.259440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:21.259440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:21.259440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:21.259440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:21.259440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:21.259440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:21.259440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:59:21.259440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:21.260083 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 169/327] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 02:59:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:59:21.266387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62d631-d0da-47cd-9156-fce9b2b5d73c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:21.300781 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:21.384289 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798d6f1-1236-4340-9f31-2c12747d1ba4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:21.396809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62d631-d0da-47cd-9156-fce9b2b5d73c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:21.417151 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:21.427488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc0e8a-0b25-4c01-ae76-1cecf9e8009d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:21.427488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc0e8a-0b25-4c01-ae76-1cecf9e8009d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:21.427488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc0e8a-0b25-4c01-ae76-1cecf9e8009d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:21.427865 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc0e8a-0b25-4c01-ae76-1cecf9e8009d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:21.439949 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01443e92-82be-4046-9563-4dad68388a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:21.440695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01443e92-82be-4046-9563-4dad68388a94 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:21.440899 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01443e92-82be-4046-9563-4dad68388a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:21.441176 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01443e92-82be-4046-9563-4dad68388a94 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:21.442390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01443e92-82be-4046-9563-4dad68388a94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:21.442691 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01443e92-82be-4046-9563-4dad68388a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:21.444099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01443e92-82be-4046-9563-4dad68388a94 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:21.453559 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 170/328] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:21.459824 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01443e92-82be-4046-9563-4dad68388a94 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:21.511847 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc0e8a-0b25-4c01-ae76-1cecf9e8009d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:21.513357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc0e8a-0b25-4c01-ae76-1cecf9e8009d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:21.554915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:21.579432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01443e92-82be-4046-9563-4dad68388a94 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:21.620562 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb202eb-59b6-44cf-9c31-39c565e7594a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:21.621299 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb202eb-59b6-44cf-9c31-39c565e7594a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:21.621574 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb202eb-59b6-44cf-9c31-39c565e7594a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:21.621841 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb202eb-59b6-44cf-9c31-39c565e7594a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:21.639724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 159/329] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:59:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 02:59:21.646123 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc0e8a-0b25-4c01-ae76-1cecf9e8009d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:21.680097 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:21.706320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb202eb-59b6-44cf-9c31-39c565e7594a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:21.707806 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb202eb-59b6-44cf-9c31-39c565e7594a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:21.764313 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc0e8a-0b25-4c01-ae76-1cecf9e8009d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:21.804113 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c42395-d18b-4bf5-95ea-acbad385f03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:21.804811 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c42395-d18b-4bf5-95ea-acbad385f03a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:21.805096 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c42395-d18b-4bf5-95ea-acbad385f03a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:21.805369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c42395-d18b-4bf5-95ea-acbad385f03a None admin] PATH_INFO: `/v3/projects/615d529fe91a4c82b4d4fe25f10a0478` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:21.806776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c42395-d18b-4bf5-95ea-acbad385f03a None admin] RBAC: Authorizing `identity:get_project(project_id=615d529fe91a4c82b4d4fe25f10a0478)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:21.808335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c42395-d18b-4bf5-95ea-acbad385f03a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:21.810270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 160/330] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:59:21 2022] GET /identity/v3/projects/615d529fe91a4c82b4d4fe25f10a0478 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:21.815576 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c42395-d18b-4bf5-95ea-acbad385f03a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:21.819685 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:21.831643 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 171/331] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:59:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 02:59:21.838015 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb202eb-59b6-44cf-9c31-39c565e7594a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:21.915420 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:21.917553 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c42395-d18b-4bf5-95ea-acbad385f03a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:21.935380 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb202eb-59b6-44cf-9c31-39c565e7594a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:21.950882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427c301-b224-46d0-8435-0cc0e703158d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:21.951520 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427c301-b224-46d0-8435-0cc0e703158d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:21.951800 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427c301-b224-46d0-8435-0cc0e703158d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:21.952079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427c301-b224-46d0-8435-0cc0e703158d None admin] PATH_INFO: `/v3/groups/68a2a5a679e94871a407cedec669a34b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:21.953180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427c301-b224-46d0-8435-0cc0e703158d None admin] RBAC: Authorizing `identity:get_group(group_id=68a2a5a679e94871a407cedec669a34b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:21.954639 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427c301-b224-46d0-8435-0cc0e703158d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:21.956310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 161/332] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Oct 17 02:59:21 2022] GET /identity/v3/groups/68a2a5a679e94871a407cedec669a34b => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:21.962747 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427c301-b224-46d0-8435-0cc0e703158d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:21.969117 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0361c-bcd4-4380-9bb3-82e5d5f688ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:21.969749 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0361c-bcd4-4380-9bb3-82e5d5f688ac None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:21.970032 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0361c-bcd4-4380-9bb3-82e5d5f688ac None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:21.970303 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0361c-bcd4-4380-9bb3-82e5d5f688ac None admin] PATH_INFO: `/v3/projects/eb6be797d9e4443fad02ceee69bd83a7` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:21.971444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0361c-bcd4-4380-9bb3-82e5d5f688ac None admin] RBAC: Authorizing `identity:get_project(project_id=eb6be797d9e4443fad02ceee69bd83a7)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:21.973455 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0361c-bcd4-4380-9bb3-82e5d5f688ac None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:21.979365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 172/333] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 02:59:21 2022] GET /identity/v3/projects/eb6be797d9e4443fad02ceee69bd83a7 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:21.980509 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0361c-bcd4-4380-9bb3-82e5d5f688ac None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:22.032974 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:22.062643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427c301-b224-46d0-8435-0cc0e703158d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:22.077488 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0361c-bcd4-4380-9bb3-82e5d5f688ac None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:22.096018 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f78307-7e3b-4587-9911-80dbf99e910f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:22.096610 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f78307-7e3b-4587-9911-80dbf99e910f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:22.096887 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f78307-7e3b-4587-9911-80dbf99e910f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:22.097159 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f78307-7e3b-4587-9911-80dbf99e910f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:22.098404 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f78307-7e3b-4587-9911-80dbf99e910f None admin] RBAC: Adding query filter params (group.id=68a2a5a679e94871a407cedec669a34b, role.id=baf207ce98e448f1a2e10c3310a0e33b, scope.project.id=615d529fe91a4c82b4d4fe25f10a0478) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:22.098715 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f78307-7e3b-4587-9911-80dbf99e910f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:22.100170 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f78307-7e3b-4587-9911-80dbf99e910f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:22.105749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 162/334] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Oct 17 02:59:21 2022] GET /identity/v3/role_assignments?group.id=68a2a5a679e94871a407cedec669a34b&scope.project.id=615d529fe91a4c82b4d4fe25f10a0478&role.id=baf207ce98e448f1a2e10c3310a0e33b => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:22.111524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a138479-dc95-4872-a892-6c0a726c1eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:22.112179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a138479-dc95-4872-a892-6c0a726c1eca None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:22.112465 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a138479-dc95-4872-a892-6c0a726c1eca None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:22.112741 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a138479-dc95-4872-a892-6c0a726c1eca None admin] PATH_INFO: `/v3/groups/93f70af99a1a4c929be06c49c74f96d4` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:22.113990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a138479-dc95-4872-a892-6c0a726c1eca None admin] RBAC: Authorizing `identity:get_group(group_id=93f70af99a1a4c929be06c49c74f96d4)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:22.115505 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a138479-dc95-4872-a892-6c0a726c1eca None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:22.117179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 173/335] 10.222.0.27 () {56 vars in 1180 bytes} [Mon Oct 17 02:59:21 2022] GET /identity/v3/groups/93f70af99a1a4c929be06c49c74f96d4 => generated 229 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:22.123544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f78307-7e3b-4587-9911-80dbf99e910f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:22.229838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f78307-7e3b-4587-9911-80dbf99e910f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:22.264912 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe3cdf-d956-4ce3-9926-fb96f72cfdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:22.265586 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe3cdf-d956-4ce3-9926-fb96f72cfdfa None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:22.265798 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe3cdf-d956-4ce3-9926-fb96f72cfdfa None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:22.266077 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe3cdf-d956-4ce3-9926-fb96f72cfdfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:22.266813 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe3cdf-d956-4ce3-9926-fb96f72cfdfa None admin] RBAC: Adding query filter params (group.id=93f70af99a1a4c929be06c49c74f96d4, role.id=ea0fd0c10f2f4e529270edbe3525d4c6, scope.project.id=eb6be797d9e4443fad02ceee69bd83a7) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:22.267155 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe3cdf-d956-4ce3-9926-fb96f72cfdfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:22.268594 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe3cdf-d956-4ce3-9926-fb96f72cfdfa None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:22.272160 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:22.279150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 163/336] 10.222.0.27 () {56 vars in 1376 bytes} [Mon Oct 17 02:59:22 2022] GET /identity/v3/role_assignments?group.id=93f70af99a1a4c929be06c49c74f96d4&scope.project.id=eb6be797d9e4443fad02ceee69bd83a7&role.id=ea0fd0c10f2f4e529270edbe3525d4c6 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:24.998350 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8a6eb-be38-4567-aa25-2d1d07218ee5 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:24.999421 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8a6eb-be38-4567-aa25-2d1d07218ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:24.999927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8a6eb-be38-4567-aa25-2d1d07218ee5 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:25.001749 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 174/337] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:25.007305 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746ad05-ddb7-4060-bfb3-138e5d6724f4 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:25.008127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746ad05-ddb7-4060-bfb3-138e5d6724f4 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:25.008622 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746ad05-ddb7-4060-bfb3-138e5d6724f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:25.031437 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1746ad05-ddb7-4060-bfb3-138e5d6724f4 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:25.032953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1746ad05-ddb7-4060-bfb3-138e5d6724f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:25.119826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 164/338] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:25.291358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7027e86-dacc-4353-a6a5-fcb11df71f1d None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:25.291938 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7027e86-dacc-4353-a6a5-fcb11df71f1d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:25.292417 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7027e86-dacc-4353-a6a5-fcb11df71f1d None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:25.294226 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 175/339] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:25.299192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a98eed-336b-4918-8928-0fd95f6b0579 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:25.299727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a98eed-336b-4918-8928-0fd95f6b0579 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:25.300219 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a98eed-336b-4918-8928-0fd95f6b0579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:25.323070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-80a98eed-336b-4918-8928-0fd95f6b0579 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:25.324587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-80a98eed-336b-4918-8928-0fd95f6b0579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:25.412431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6908cf-7218-4628-9755-804331990b73 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:25.413069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6908cf-7218-4628-9755-804331990b73 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:25.413536 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6908cf-7218-4628-9755-804331990b73 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:25.415088 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 176/340] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:25.418770 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 165/341] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:25.423128 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c8914-c547-4ac3-81ff-cab3ef2fa939 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:25.423633 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c8914-c547-4ac3-81ff-cab3ef2fa939 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:25.424112 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c8914-c547-4ac3-81ff-cab3ef2fa939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:25.451851 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ae7c8914-c547-4ac3-81ff-cab3ef2fa939 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:25.453381 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ae7c8914-c547-4ac3-81ff-cab3ef2fa939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:25.468446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:25.482192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a98eed-336b-4918-8928-0fd95f6b0579 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:25.559237 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:25.563250 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 177/342] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:59:25.622589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a98eed-336b-4918-8928-0fd95f6b0579 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:25.667641 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fdcc6-895b-4551-81a3-08e173e88b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:25.668603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fdcc6-895b-4551-81a3-08e173e88b30 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:25.669017 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fdcc6-895b-4551-81a3-08e173e88b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:25.669429 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fdcc6-895b-4551-81a3-08e173e88b30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:25.688703 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fdcc6-895b-4551-81a3-08e173e88b30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:25.690354 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fdcc6-895b-4551-81a3-08e173e88b30 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:25.700001 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9c1f5-d91e-4c16-8b72-e9988ace5154 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:25.700001 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9c1f5-d91e-4c16-8b72-e9988ace5154 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:25.700001 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9c1f5-d91e-4c16-8b72-e9988ace5154 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:25.700493 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 178/343] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 02:59:25.705472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d786806-20a1-426f-b6f7-d7f7be2a5a0e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:25.705758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d786806-20a1-426f-b6f7-d7f7be2a5a0e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:25.706034 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d786806-20a1-426f-b6f7-d7f7be2a5a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:25.709755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-892fdcc6-895b-4551-81a3-08e173e88b30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:25.709755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:25.709755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:25.709755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:25.709755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:25.709755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:25.709755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:25.709755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:25.709755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:25.709755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:25.709755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:25.709755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:25.709755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:25.709755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:25.709755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:25.709755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:25.710915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:25.710915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:25.710915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:25.710915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:25.710915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:25.710915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:25.710915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:25.710915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:25.710915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:25.710915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:25.710915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:25.710915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:25.710915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:25.710915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:25.710915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:25.710915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:25.710915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:25.712028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:25.712028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:25.712028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:25.713681 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 166/344] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:59:25.719111 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fdcc6-895b-4551-81a3-08e173e88b30 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:25.742042 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8d786806-20a1-426f-b6f7-d7f7be2a5a0e None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:25.744242 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8d786806-20a1-426f-b6f7-d7f7be2a5a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:25.826848 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 179/345] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:25.827747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d786806-20a1-426f-b6f7-d7f7be2a5a0e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:25.844581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fdcc6-895b-4551-81a3-08e173e88b30 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:25.858162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:25.889860 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea084d0-7b26-46d8-aa19-660906224415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:25.890568 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea084d0-7b26-46d8-aa19-660906224415 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:25.891000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea084d0-7b26-46d8-aa19-660906224415 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:25.891352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea084d0-7b26-46d8-aa19-660906224415 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:25.892073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea084d0-7b26-46d8-aa19-660906224415 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:25.892444 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea084d0-7b26-46d8-aa19-660906224415 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:25.894028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea084d0-7b26-46d8-aa19-660906224415 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:25.912346 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 167/346] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:25.917656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea084d0-7b26-46d8-aa19-660906224415 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:25.921153 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:25.949812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d786806-20a1-426f-b6f7-d7f7be2a5a0e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:25.986549 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10342c3-c80f-4f5e-8062-efdea02fea3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:25.987233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10342c3-c80f-4f5e-8062-efdea02fea3d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:25.987516 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10342c3-c80f-4f5e-8062-efdea02fea3d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:25.987795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10342c3-c80f-4f5e-8062-efdea02fea3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:26.001359 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10342c3-c80f-4f5e-8062-efdea02fea3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:26.002812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10342c3-c80f-4f5e-8062-efdea02fea3d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:26.019334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-c10342c3-c80f-4f5e-8062-efdea02fea3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:26.019334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:26.019334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:26.019334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:26.019334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:26.019334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:26.019334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:26.019334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:26.019334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:26.019334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:26.019334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:26.019334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:26.019334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:26.019334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:26.019334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:26.019334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:26.022035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:26.022035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:26.022035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:26.022035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:26.022035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:26.022035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:26.022035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:26.022035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:26.022035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:26.022035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:26.022035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:26.022035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:26.022035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:26.022035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:26.022035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:26.022035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:26.022035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:26.022931 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:26.022931 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:26.022931 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:26.022931 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 180/347] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:26.022931 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10342c3-c80f-4f5e-8062-efdea02fea3d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:26.027462 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea084d0-7b26-46d8-aa19-660906224415 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:26.063855 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d86e1-5133-4b2f-af08-3078f2ebd5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:26.064477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d86e1-5133-4b2f-af08-3078f2ebd5c3 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:26.064756 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d86e1-5133-4b2f-af08-3078f2ebd5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:26.065031 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d86e1-5133-4b2f-af08-3078f2ebd5c3 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:26.066109 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d86e1-5133-4b2f-af08-3078f2ebd5c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:26.068482 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d86e1-5133-4b2f-af08-3078f2ebd5c3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:26.096834 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 168/348] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 02:59:25 2022] POST /identity/v3/users => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 02:59:26.099374 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d86e1-5133-4b2f-af08-3078f2ebd5c3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:26.148373 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:26.162307 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10342c3-c80f-4f5e-8062-efdea02fea3d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:26.197665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:26.208246 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4588435-480e-489f-aee3-ec2b40e089fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:26.209986 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4588435-480e-489f-aee3-ec2b40e089fd None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:26.210491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4588435-480e-489f-aee3-ec2b40e089fd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:26.211006 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4588435-480e-489f-aee3-ec2b40e089fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:26.225308 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4588435-480e-489f-aee3-ec2b40e089fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:26.229416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4588435-480e-489f-aee3-ec2b40e089fd None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:26.248061 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d86e1-5133-4b2f-af08-3078f2ebd5c3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:26.248693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-c4588435-480e-489f-aee3-ec2b40e089fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:26.248693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:26.248693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:26.248693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:26.248693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:26.248693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:26.248693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:26.248693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:26.248693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:26.248693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:26.248693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:26.248693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:26.248693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:26.248693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:26.248693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:26.248693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:26.249570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:26.249570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:26.249570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:26.249570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:26.249570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:26.249570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:26.249570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:26.249570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:26.249570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:26.249570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:26.249570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:26.249570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:26.249570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:26.249570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:26.249570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:26.249570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:26.249570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:26.251025 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:26.251025 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:26.251025 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:26.251025 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 181/349] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:26.251278 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4588435-480e-489f-aee3-ec2b40e089fd None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:26.298120 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536cd55-3656-462b-970e-bb44225ae910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:26.299739 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536cd55-3656-462b-970e-bb44225ae910 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:26.300034 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536cd55-3656-462b-970e-bb44225ae910 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:26.300435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536cd55-3656-462b-970e-bb44225ae910 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:26.301245 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536cd55-3656-462b-970e-bb44225ae910 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:26.301685 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536cd55-3656-462b-970e-bb44225ae910 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:26.303180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536cd55-3656-462b-970e-bb44225ae910 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:26.319142 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 169/350] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:26.319923 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536cd55-3656-462b-970e-bb44225ae910 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:26.392469 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4588435-480e-489f-aee3-ec2b40e089fd None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:26.431044 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:26.439304 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa50cd-4e79-4d89-b74f-aca45d987503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:26.440116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa50cd-4e79-4d89-b74f-aca45d987503 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:26.440403 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa50cd-4e79-4d89-b74f-aca45d987503 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:26.440774 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa50cd-4e79-4d89-b74f-aca45d987503 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:26.445180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536cd55-3656-462b-970e-bb44225ae910 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:26.451575 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa50cd-4e79-4d89-b74f-aca45d987503 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:26.453033 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa50cd-4e79-4d89-b74f-aca45d987503 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:26.467712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-5dfa50cd-4e79-4d89-b74f-aca45d987503 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:26.467712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:26.467712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:26.467712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:26.467712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:26.467712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:26.467712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:26.467712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:26.467712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:26.467712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:26.467712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:26.467712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:26.467712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:26.467712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:26.467712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:26.467712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:26.468655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:26.468655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:26.468655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:26.468655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:26.468655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:26.468655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:26.468655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:26.468655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:26.468655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:26.468655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:26.468655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:26.468655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:26.468655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:26.468655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:26.468655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:26.468655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:26.468655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:26.469522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:26.469522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:26.469522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:26.469522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 182/351] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:26.469751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa50cd-4e79-4d89-b74f-aca45d987503 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:26.491508 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac71d6-f492-4937-a3f6-82d913e036ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:26.492131 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac71d6-f492-4937-a3f6-82d913e036ae None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:26.492407 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac71d6-f492-4937-a3f6-82d913e036ae None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:26.492676 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac71d6-f492-4937-a3f6-82d913e036ae None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:26.493873 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac71d6-f492-4937-a3f6-82d913e036ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:26.494181 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac71d6-f492-4937-a3f6-82d913e036ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:26.496045 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac71d6-f492-4937-a3f6-82d913e036ae None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:26.502852 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:26.510320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 170/352] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:26.510947 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a036b8-236e-42cf-b5e3-591c6cfb9b73 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:26.511236 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a036b8-236e-42cf-b5e3-591c6cfb9b73 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:26.511510 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a036b8-236e-42cf-b5e3-591c6cfb9b73 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:26.512878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 171/353] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:26.513393 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a036b8-236e-42cf-b5e3-591c6cfb9b73 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:26.588408 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa50cd-4e79-4d89-b74f-aca45d987503 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:26.623570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fa036-1277-4e04-87c6-d581bbcc9805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:26.624060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a036b8-236e-42cf-b5e3-591c6cfb9b73 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:26.624212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fa036-1277-4e04-87c6-d581bbcc9805 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:26.624475 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fa036-1277-4e04-87c6-d581bbcc9805 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:26.624754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fa036-1277-4e04-87c6-d581bbcc9805 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:26.625409 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18fa036-1277-4e04-87c6-d581bbcc9805 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:26.627952 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18fa036-1277-4e04-87c6-d581bbcc9805 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:26.652532 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 183/354] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 02:59:26 2022] POST /identity/v3/users => generated 248 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:59:26.653252 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fa036-1277-4e04-87c6-d581bbcc9805 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:26.670186 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7207b-b3c2-4fac-a5da-7af89f66ad8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:26.671363 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7207b-b3c2-4fac-a5da-7af89f66ad8c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:26.671906 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7207b-b3c2-4fac-a5da-7af89f66ad8c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:26.672377 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7207b-b3c2-4fac-a5da-7af89f66ad8c None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:26.673303 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7207b-b3c2-4fac-a5da-7af89f66ad8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:26.674215 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7207b-b3c2-4fac-a5da-7af89f66ad8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:26.676165 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7207b-b3c2-4fac-a5da-7af89f66ad8c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:26.691283 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 172/355] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:26.693607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44867f9e-8e76-433d-b6fa-dee405a9b688 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:26.694162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44867f9e-8e76-433d-b6fa-dee405a9b688 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:26.694659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44867f9e-8e76-433d-b6fa-dee405a9b688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:26.715222 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-44867f9e-8e76-433d-b6fa-dee405a9b688 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:26.716956 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-44867f9e-8e76-433d-b6fa-dee405a9b688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:26.719686 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:26.778756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:26.796611 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fa036-1277-4e04-87c6-d581bbcc9805 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:26.819346 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 173/356] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:26.820440 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44867f9e-8e76-433d-b6fa-dee405a9b688 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:26.838906 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fe516-7349-4388-8cf4-bd64b9ead926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:26.839913 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe516-7349-4388-8cf4-bd64b9ead926 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:26.840392 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe516-7349-4388-8cf4-bd64b9ead926 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:26.840877 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe516-7349-4388-8cf4-bd64b9ead926 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:26.841721 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fe516-7349-4388-8cf4-bd64b9ead926 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:26.843395 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fe516-7349-4388-8cf4-bd64b9ead926 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:26.871695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 184/357] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 02:59:26 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:59:26.941079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44867f9e-8e76-433d-b6fa-dee405a9b688 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:26.982375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2266e-30a5-48e4-b540-47b8b32412c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:26.983102 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2266e-30a5-48e4-b540-47b8b32412c2 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:26.983388 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2266e-30a5-48e4-b540-47b8b32412c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:26.983694 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2266e-30a5-48e4-b540-47b8b32412c2 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:26.984363 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2266e-30a5-48e4-b540-47b8b32412c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:26.985839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2266e-30a5-48e4-b540-47b8b32412c2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:26.997499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:27.023124 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 174/358] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 02:59:26 2022] POST /identity/v3/users => generated 249 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:59:27.148969 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fe516-7349-4388-8cf4-bd64b9ead926 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:27.271859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fe516-7349-4388-8cf4-bd64b9ead926 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:27.313116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcc7fd-3b45-4b0a-bfc4-ad492437f401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:27.313728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcc7fd-3b45-4b0a-bfc4-ad492437f401 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:27.314007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcc7fd-3b45-4b0a-bfc4-ad492437f401 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:27.314278 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcc7fd-3b45-4b0a-bfc4-ad492437f401 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:27.314987 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcc7fd-3b45-4b0a-bfc4-ad492437f401 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:27.316355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcc7fd-3b45-4b0a-bfc4-ad492437f401 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:27.327754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.sql.core [None req-d4bcc7fd-3b45-4b0a-bfc4-ad492437f401 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 02:59:27.327754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 17 02:59:27.327754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [parameters: {'id': '3236bb4d1b11474b987ad41363660fb9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 17 02:59:27.327754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81582) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 02:59:27.336234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d4bcc7fd-3b45-4b0a-bfc4-ad492437f401 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 17 02:59:27.336234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:27.336234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 02:59:27.336234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 02:59:27.336234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 02:59:27.336234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 02:59:27.336234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 02:59:27.336234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 02:59:27.336234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 02:59:27.336234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application conn.query(q) Oct 17 02:59:27.336234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 02:59:27.336234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 02:59:27.336234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 02:59:27.336234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result.read() Oct 17 02:59:27.336234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return ref.to_dict() Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 02:59:27.337106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application next(self.gen) Oct 17 02:59:27.338023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 02:59:27.338023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application yield resource Oct 17 02:59:27.338023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 02:59:27.338023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application next(self.gen) Oct 17 02:59:27.338023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 02:59:27.338023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 02:59:27.338023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 02:59:27.338023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 02:59:27.338023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 02:59:27.338023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise self.value Oct 17 02:59:27.338023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 02:59:27.338023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 02:59:27.338023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 02:59:27.338023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application session.commit() Oct 17 02:59:27.338023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 02:59:27.338023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 02:59:27.338023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 02:59:27.338955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 02:59:27.338955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 02:59:27.338955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.session.flush() Oct 17 02:59:27.338955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 02:59:27.338955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 02:59:27.338955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 02:59:27.338955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 02:59:27.338955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 02:59:27.338955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application compat.raise_( Oct 17 02:59:27.338955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 02:59:27.338955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception Oct 17 02:59:27.338955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 02:59:27.338955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 02:59:27.338955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 02:59:27.338955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 02:59:27.338955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 02:59:27.338955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 02:59:27.339857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 02:59:27.339857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 02:59:27.339857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 02:59:27.339857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 02:59:27.339857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 02:59:27.339857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 02:59:27.339857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 02:59:27.339857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 02:59:27.339857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 02:59:27.339857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 02:59:27.339857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 02:59:27.339857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 02:59:27.339857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 02:59:27.339857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 02:59:27.339857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 02:59:27.339857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception Oct 17 02:59:27.342113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 02:59:27.342113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 02:59:27.342113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 02:59:27.342113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 02:59:27.342113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 02:59:27.342113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 02:59:27.342113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 02:59:27.342113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application conn.query(q) Oct 17 02:59:27.342113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 02:59:27.342113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 02:59:27.342113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 02:59:27.342113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result.read() Oct 17 02:59:27.342113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 02:59:27.342113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 02:59:27.342113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 02:59:27.342113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 02:59:27.347508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 02:59:27.347508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 02:59:27.347508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 02:59:27.347508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 02:59:27.347508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 02:59:27.347508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 17 02:59:27.347508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [parameters: {'id': '3236bb4d1b11474b987ad41363660fb9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 17 02:59:27.347508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 02:59:27.347508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:27.347508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 02:59:27.347508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:27.347508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:27.347508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:27.347508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:27.347508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:27.347508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:27.349503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:27.349503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:27.349503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:27.349503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:27.349503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:27.349503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:27.349503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 17 02:59:27.349503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 17 02:59:27.349503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:27.349503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:27.349503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 17 02:59:27.349503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 17 02:59:27.349503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 02:59:27.349503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 02:59:27.349503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 17 02:59:27.349503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:27.350526 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 185/359] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Oct 17 02:59:27 2022] POST /identity/v3/roles => generated 144 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:27.353127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2266e-30a5-48e4-b540-47b8b32412c2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:27.462297 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2266e-30a5-48e4-b540-47b8b32412c2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:27.497688 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6382920-066f-4ca8-8891-671e32e1eaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:27.498392 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6382920-066f-4ca8-8891-671e32e1eaae None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:27.498669 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6382920-066f-4ca8-8891-671e32e1eaae None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:27.498965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6382920-066f-4ca8-8891-671e32e1eaae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:27.510373 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6382920-066f-4ca8-8891-671e32e1eaae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:27.515717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-d6382920-066f-4ca8-8891-671e32e1eaae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:59:27.515717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:27.515717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:27.515717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:27.515717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:27.515717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:27.515717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:27.515717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:27.515717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:27.515717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:27.515717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:27.515717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:27.515717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:27.515717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:27.515717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:27.515717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:27.516612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:27.516612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:27.516612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:27.516612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:27.516612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:27.516612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:27.516612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:27.516612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:27.516612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:27.516612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:27.516612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:27.516612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:27.516612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:27.516612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:27.516612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:27.516612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:27.517464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:27.517464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:27.517464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:27.517464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:27.517464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:27.517464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:27.517464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:27.517464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:27.517464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:27.517464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:27.517464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 02:59:27.517464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:27.517464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 175/360] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Oct 17 02:59:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:59:27.528019 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcc7fd-3b45-4b0a-bfc4-ad492437f401 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:27.543208 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:27.659536 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcc7fd-3b45-4b0a-bfc4-ad492437f401 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:27.711911 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d703f5-97d9-4aec-8d76-a490d9c0d826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:27.712892 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d703f5-97d9-4aec-8d76-a490d9c0d826 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:27.713391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d703f5-97d9-4aec-8d76-a490d9c0d826 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:27.713885 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d703f5-97d9-4aec-8d76-a490d9c0d826 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:27.714748 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d703f5-97d9-4aec-8d76-a490d9c0d826 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:27.715295 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d703f5-97d9-4aec-8d76-a490d9c0d826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:27.716896 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d703f5-97d9-4aec-8d76-a490d9c0d826 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:27.730244 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 186/361] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:28.477633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c6564-b601-49cd-90bf-cd029f9ea57b None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:28.478167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c6564-b601-49cd-90bf-cd029f9ea57b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:28.478167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c6564-b601-49cd-90bf-cd029f9ea57b None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:28.479790 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 176/362] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:28.485156 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6403d2-f5e2-4aac-ac70-32e5f16ab6ac None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:28.485543 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6403d2-f5e2-4aac-ac70-32e5f16ab6ac None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:28.485707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6403d2-f5e2-4aac-ac70-32e5f16ab6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:28.511837 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ee6403d2-f5e2-4aac-ac70-32e5f16ab6ac None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:28.513037 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ee6403d2-f5e2-4aac-ac70-32e5f16ab6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:28.593073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:28.598724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 187/363] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:28.949746 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c6564-b601-49cd-90bf-cd029f9ea57b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:29.018016 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:29.083215 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c6564-b601-49cd-90bf-cd029f9ea57b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:29.126965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f0fdc-52d0-4c84-9113-665383d07f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:29.127605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f0fdc-52d0-4c84-9113-665383d07f8e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:29.127882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f0fdc-52d0-4c84-9113-665383d07f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:29.128149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f0fdc-52d0-4c84-9113-665383d07f8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:29.147765 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f0fdc-52d0-4c84-9113-665383d07f8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:29.152674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-e60f0fdc-52d0-4c84-9113-665383d07f8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:29.152674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:29.152674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:29.152674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:29.152674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:29.152674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:29.152674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:29.152674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:29.152674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:29.152674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:29.152674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:29.152674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:29.152674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:29.152674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:29.152674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:29.152674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:29.153663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:29.153663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:29.153663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:29.153663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:29.153663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:29.153663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:29.153663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:29.153663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:29.153663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:29.153663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:29.153663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:29.153663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:29.153663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:29.153663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:29.153663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:29.153663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:29.154481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:29.154481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:29.154481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:29.154481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:29.154481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:29.154481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:29.154481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:29.154481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:29.154481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:29.154481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:29.154481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:29.154481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:29.154481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 177/364] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Oct 17 02:59:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:29.159771 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6403d2-f5e2-4aac-ac70-32e5f16ab6ac None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:29.312232 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6403d2-f5e2-4aac-ac70-32e5f16ab6ac None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:29.360850 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768af8d0-88d9-4b84-9dd6-d0f4ad853bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:29.361466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768af8d0-88d9-4b84-9dd6-d0f4ad853bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:29.361743 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768af8d0-88d9-4b84-9dd6-d0f4ad853bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:29.362010 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768af8d0-88d9-4b84-9dd6-d0f4ad853bc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:29.362673 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768af8d0-88d9-4b84-9dd6-d0f4ad853bc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:29.362994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768af8d0-88d9-4b84-9dd6-d0f4ad853bc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:29.364918 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768af8d0-88d9-4b84-9dd6-d0f4ad853bc3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:29.376338 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 188/365] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Oct 17 02:59:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:29.382693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f0fdc-52d0-4c84-9113-665383d07f8e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:29.407109 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb80289-a5e1-430a-a0fe-15ce7e554af6 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:29.407438 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb80289-a5e1-430a-a0fe-15ce7e554af6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:29.407664 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb80289-a5e1-430a-a0fe-15ce7e554af6 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:29.409060 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 189/366] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 02:59:29.420414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca67ba-765c-40ec-8723-dea8fcc165c9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:29.420707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca67ba-765c-40ec-8723-dea8fcc165c9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:29.420956 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca67ba-765c-40ec-8723-dea8fcc165c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:29.459248 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3eca67ba-765c-40ec-8723-dea8fcc165c9 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:29.460460 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3eca67ba-765c-40ec-8723-dea8fcc165c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:29.503285 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f0fdc-52d0-4c84-9113-665383d07f8e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:29.518633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:29.558761 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0aa61d-63e1-4932-8cf8-aa61e18af5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:29.559439 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0aa61d-63e1-4932-8cf8-aa61e18af5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:29.559721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0aa61d-63e1-4932-8cf8-aa61e18af5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:29.559994 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0aa61d-63e1-4932-8cf8-aa61e18af5a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:29.581498 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0aa61d-63e1-4932-8cf8-aa61e18af5a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:29.583066 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0aa61d-63e1-4932-8cf8-aa61e18af5a4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:29.584785 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:29.604722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-db0aa61d-63e1-4932-8cf8-aa61e18af5a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:29.604722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:29.604722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:29.604722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:29.604722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:29.604722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:29.604722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:29.604722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:29.604722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:29.604722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:29.604722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:29.604722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:29.604722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:29.604722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:29.604722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:29.604722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:29.605826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:29.605826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:29.605826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:29.605826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:29.605826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:29.605826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:29.605826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:29.605826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:29.605826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:29.605826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:29.605826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:29.605826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:29.605826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:29.605826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:29.605826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:29.605826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:29.605826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:29.606879 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:29.606879 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:29.606879 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:29.608565 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 178/367] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:29.614348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0aa61d-63e1-4932-8cf8-aa61e18af5a4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:29.620408 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 190/368] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:59:29.766107 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0aa61d-63e1-4932-8cf8-aa61e18af5a4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:29.768625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12629be3-db57-48df-81ca-d9fe2c5b6f88 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:29.769181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12629be3-db57-48df-81ca-d9fe2c5b6f88 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:29.769671 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12629be3-db57-48df-81ca-d9fe2c5b6f88 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:29.771550 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 191/369] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 02:59:29.776595 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47aacab-2302-459f-b539-1beec763a079 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:29.777093 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47aacab-2302-459f-b539-1beec763a079 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:29.777584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47aacab-2302-459f-b539-1beec763a079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:29.797519 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a47aacab-2302-459f-b539-1beec763a079 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:29.799030 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a47aacab-2302-459f-b539-1beec763a079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:29.804584 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477ec9f-8b03-4433-ab78-ebc0733b2a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:29.805261 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477ec9f-8b03-4433-ab78-ebc0733b2a26 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:29.805543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477ec9f-8b03-4433-ab78-ebc0733b2a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:29.805818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477ec9f-8b03-4433-ab78-ebc0733b2a26 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:29.806520 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477ec9f-8b03-4433-ab78-ebc0733b2a26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:29.806825 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477ec9f-8b03-4433-ab78-ebc0733b2a26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:29.809135 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477ec9f-8b03-4433-ab78-ebc0733b2a26 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:29.825739 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 179/370] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:29.833588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477ec9f-8b03-4433-ab78-ebc0733b2a26 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:29.892753 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 192/371] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:29.916489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:29.953512 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47aacab-2302-459f-b539-1beec763a079 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:30.034152 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477ec9f-8b03-4433-ab78-ebc0733b2a26 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:30.077634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f54618-818a-4343-95e7-100a655a5b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:30.079031 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f54618-818a-4343-95e7-100a655a5b97 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:30.079532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f54618-818a-4343-95e7-100a655a5b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:30.080009 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f54618-818a-4343-95e7-100a655a5b97 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:30.090717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f54618-818a-4343-95e7-100a655a5b97 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:30.093088 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f54618-818a-4343-95e7-100a655a5b97 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:30.094162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47aacab-2302-459f-b539-1beec763a079 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:30.110772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-17f54618-818a-4343-95e7-100a655a5b97 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 02:59:30.110772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:30.110772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:30.110772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:30.110772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:30.110772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:30.110772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:30.110772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:30.110772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:30.110772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:30.110772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:30.110772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:30.110772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 02:59:30.110772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 02:59:30.110772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 02:59:30.110772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:30.111766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:30.112659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:30.112659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:30.112659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 02:59:30.112659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 02:59:30.112659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 02:59:30.112659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 02:59:30.112659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 02:59:30.112659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 02:59:30.112659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 02:59:30.112659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:30.120050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 180/372] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Oct 17 02:59:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:30.120050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb7576-5380-4b9f-b2c0-9a245cd1e07c None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:30.120050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb7576-5380-4b9f-b2c0-9a245cd1e07c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:30.120050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb7576-5380-4b9f-b2c0-9a245cd1e07c None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:30.120050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 181/373] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:30.121330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb7576-5380-4b9f-b2c0-9a245cd1e07c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:30.135437 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:30.150036 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1fb596-8d9b-4cbf-8599-41c66612dc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:30.151032 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fb596-8d9b-4cbf-8599-41c66612dc80 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:30.152097 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fb596-8d9b-4cbf-8599-41c66612dc80 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:30.152568 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fb596-8d9b-4cbf-8599-41c66612dc80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:30.166459 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1fb596-8d9b-4cbf-8599-41c66612dc80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:30.173571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-9a1fb596-8d9b-4cbf-8599-41c66612dc80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:30.173571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:30.173571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:30.173571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:30.173571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:30.173571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:30.173571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:30.173571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:30.173571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:30.173571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:30.173571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:30.173571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:30.173571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:30.173571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:30.173571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:30.173571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:30.174496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:30.174496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:30.174496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:30.174496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:30.174496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:30.174496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:30.174496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:30.174496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:30.174496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:30.174496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:30.174496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:30.174496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:30.174496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:30.174496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:30.174496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:30.174496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:30.175344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:30.175344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:30.175344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:30.175344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:30.175344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:30.175344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:30.175344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:30.175344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:30.175344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:30.175344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:30.175344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:30.175344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:30.177127 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 193/374] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Oct 17 02:59:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:30.178745 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fe902-1582-4f3a-80cc-aa6c2762af5f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:30.179050 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fe902-1582-4f3a-80cc-aa6c2762af5f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:30.179323 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fe902-1582-4f3a-80cc-aa6c2762af5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:30.198321 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a32fe902-1582-4f3a-80cc-aa6c2762af5f None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:30.199542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a32fe902-1582-4f3a-80cc-aa6c2762af5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:30.234436 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:30.252052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb7576-5380-4b9f-b2c0-9a245cd1e07c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:30.273967 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 194/375] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:30.277877 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fe902-1582-4f3a-80cc-aa6c2762af5f None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:30.292305 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4753e-346b-4236-9443-61438c3bf16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:30.293217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4753e-346b-4236-9443-61438c3bf16e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:30.293669 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4753e-346b-4236-9443-61438c3bf16e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:30.294125 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4753e-346b-4236-9443-61438c3bf16e None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:30.295092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4753e-346b-4236-9443-61438c3bf16e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:30.295602 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4753e-346b-4236-9443-61438c3bf16e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:30.297251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4753e-346b-4236-9443-61438c3bf16e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:30.352446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 182/376] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Oct 17 02:59:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:30.353601 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4753e-346b-4236-9443-61438c3bf16e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:30.399606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fe902-1582-4f3a-80cc-aa6c2762af5f None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:30.413257 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:30.436919 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b839b58-9725-431c-8d01-7feb93f4c5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:30.437851 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b839b58-9725-431c-8d01-7feb93f4c5fe None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:30.438330 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b839b58-9725-431c-8d01-7feb93f4c5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:30.438800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b839b58-9725-431c-8d01-7feb93f4c5fe None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:30.439752 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b839b58-9725-431c-8d01-7feb93f4c5fe None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:30.440250 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b839b58-9725-431c-8d01-7feb93f4c5fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:30.442333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b839b58-9725-431c-8d01-7feb93f4c5fe None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:30.451554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 195/377] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Oct 17 02:59:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:30.452583 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b839b58-9725-431c-8d01-7feb93f4c5fe None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:30.459370 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4753e-346b-4236-9443-61438c3bf16e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:30.495628 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84cc492-bd3c-4cb7-9626-47566e9d9843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:30.496290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cc492-bd3c-4cb7-9626-47566e9d9843 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:30.496568 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cc492-bd3c-4cb7-9626-47566e9d9843 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:30.496839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cc492-bd3c-4cb7-9626-47566e9d9843 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:30.507770 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84cc492-bd3c-4cb7-9626-47566e9d9843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:30.512749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-c84cc492-bd3c-4cb7-9626-47566e9d9843 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:30.512749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:30.512749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:30.512749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:30.512749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:30.512749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:30.512749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:30.512749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:30.512749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:30.512749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:30.512749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:30.512749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:30.512749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:30.512749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:30.512749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:30.512749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:30.513713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:30.513713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:30.513713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:30.513713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:30.513713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:30.513713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:30.513713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:30.513713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:30.513713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:30.513713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:30.513713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:30.513713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:30.513713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:30.513713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:30.513713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:30.513713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:30.514517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:30.514517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:30.514517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:30.514517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:30.514517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:30.514517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:30.514517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:30.514517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:30.514517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:30.514517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:30.514517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:30.514517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:30.514517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 183/378] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Oct 17 02:59:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:30.514517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84cc492-bd3c-4cb7-9626-47566e9d9843 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:30.518751 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:30.566081 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b839b58-9725-431c-8d01-7feb93f4c5fe None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:30.606342 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b104f-a9f6-4d17-9542-eacd05a7d981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:30.607043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b104f-a9f6-4d17-9542-eacd05a7d981 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:30.607322 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b104f-a9f6-4d17-9542-eacd05a7d981 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:30.607599 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b104f-a9f6-4d17-9542-eacd05a7d981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:30.618640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b104f-a9f6-4d17-9542-eacd05a7d981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:30.620254 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b104f-a9f6-4d17-9542-eacd05a7d981 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:30.625980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84cc492-bd3c-4cb7-9626-47566e9d9843 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:30.636473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d83b104f-a9f6-4d17-9542-eacd05a7d981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:30.636473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:30.636473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:30.636473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:30.636473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:30.636473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:30.636473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:30.636473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:30.636473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:30.636473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:30.636473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:30.636473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:30.636473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:30.636473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:30.636473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:30.636473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:30.637397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:30.637397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:30.637397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:30.637397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:30.637397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:30.637397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:30.637397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:30.637397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:30.637397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:30.637397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:30.637397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:30.637397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:30.637397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:30.637397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:30.637397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:30.637397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:30.637397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:30.638276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:30.638276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:30.638276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:30.638276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 196/379] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:30.638509 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b104f-a9f6-4d17-9542-eacd05a7d981 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:30.668229 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79712bc-98f7-49bf-9dbe-86d85db63fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:30.668492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:30.668865 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79712bc-98f7-49bf-9dbe-86d85db63fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:30.669144 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79712bc-98f7-49bf-9dbe-86d85db63fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:30.669411 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79712bc-98f7-49bf-9dbe-86d85db63fe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:30.680772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79712bc-98f7-49bf-9dbe-86d85db63fe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:30.682236 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79712bc-98f7-49bf-9dbe-86d85db63fe0 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:30.696562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-a79712bc-98f7-49bf-9dbe-86d85db63fe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:30.696562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:30.696562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:30.696562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:30.696562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:30.696562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:30.696562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:30.696562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:30.696562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:30.696562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:30.696562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:30.696562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:30.696562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:30.696562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:30.696562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:30.696562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:30.697502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:30.697502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:30.697502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:30.697502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:30.697502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:30.697502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:30.697502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:30.697502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:30.697502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:30.697502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:30.697502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:30.697502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:30.697502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:30.697502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:30.697502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:30.697502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:30.697502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:30.698356 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:30.698356 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:30.698356 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:30.698356 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 184/380] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:30.698605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79712bc-98f7-49bf-9dbe-86d85db63fe0 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:30.747663 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b104f-a9f6-4d17-9542-eacd05a7d981 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:30.753000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:30.782317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc46c6-f88b-4ad1-a7c8-411aa0a8c607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:30.782957 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc46c6-f88b-4ad1-a7c8-411aa0a8c607 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:30.783248 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc46c6-f88b-4ad1-a7c8-411aa0a8c607 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:30.783525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc46c6-f88b-4ad1-a7c8-411aa0a8c607 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:30.784190 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc46c6-f88b-4ad1-a7c8-411aa0a8c607 None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:30.784532 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc46c6-f88b-4ad1-a7c8-411aa0a8c607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:30.786148 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc46c6-f88b-4ad1-a7c8-411aa0a8c607 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:30.794552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 197/381] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Oct 17 02:59:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:30.795394 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc46c6-f88b-4ad1-a7c8-411aa0a8c607 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:30.796068 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79712bc-98f7-49bf-9dbe-86d85db63fe0 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:30.830222 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65df281-fadc-4d68-a503-c1677984356f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:30.830846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65df281-fadc-4d68-a503-c1677984356f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:30.831159 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65df281-fadc-4d68-a503-c1677984356f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:30.831429 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65df281-fadc-4d68-a503-c1677984356f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:30.841388 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65df281-fadc-4d68-a503-c1677984356f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:30.846634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-d65df281-fadc-4d68-a503-c1677984356f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:30.846634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:30.846634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:30.846634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:30.846634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:30.846634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:30.846634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:30.846634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:30.846634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:30.846634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:30.846634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:30.846634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:30.846634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:30.846634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:30.846634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:30.846634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:30.847668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:30.847668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:30.847668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:30.847668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:30.847668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:30.847668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:30.847668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:30.847668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:30.847668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:30.847668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:30.847668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:30.847668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:30.847668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:30.847668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:30.847668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:30.847668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:30.848555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:30.848555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:30.848555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:30.848555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:30.848555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:30.848555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:30.848555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:30.848555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:30.848555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:30.848555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:30.848555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:30.848555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:30.848555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 185/382] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Oct 17 02:59:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:30.849216 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6de0f0-e345-4ee3-862d-019878ab498f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:30.849372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6de0f0-e345-4ee3-862d-019878ab498f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:30.849626 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6de0f0-e345-4ee3-862d-019878ab498f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:30.851017 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 186/383] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:30.851618 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6de0f0-e345-4ee3-862d-019878ab498f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:30.888516 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:30.901224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc46c6-f88b-4ad1-a7c8-411aa0a8c607 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:30.934756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3815ad0-a918-45bb-b59d-674300171ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:30.935391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3815ad0-a918-45bb-b59d-674300171ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:30.935677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3815ad0-a918-45bb-b59d-674300171ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:30.935953 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3815ad0-a918-45bb-b59d-674300171ef1 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:30.936594 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3815ad0-a918-45bb-b59d-674300171ef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:30.936899 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3815ad0-a918-45bb-b59d-674300171ef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:30.938277 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3815ad0-a918-45bb-b59d-674300171ef1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:30.949960 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 198/384] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:30.949960 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3815ad0-a918-45bb-b59d-674300171ef1 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:30.954650 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6de0f0-e345-4ee3-862d-019878ab498f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:30.969837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:30.987553 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e25f6-eb1c-4ce8-9322-44df6cb713c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:30.988158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e25f6-eb1c-4ce8-9322-44df6cb713c8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:30.989002 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e25f6-eb1c-4ce8-9322-44df6cb713c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:30.989243 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e25f6-eb1c-4ce8-9322-44df6cb713c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:30.989887 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0e25f6-eb1c-4ce8-9322-44df6cb713c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:30.990184 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0e25f6-eb1c-4ce8-9322-44df6cb713c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:30.991589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0e25f6-eb1c-4ce8-9322-44df6cb713c8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:31.002761 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 187/385] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:31.003985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64b1d5-148a-4149-8e6c-6f4a28cb1b63 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:31.004391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64b1d5-148a-4149-8e6c-6f4a28cb1b63 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:31.004671 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64b1d5-148a-4149-8e6c-6f4a28cb1b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:31.022429 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3c64b1d5-148a-4149-8e6c-6f4a28cb1b63 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:31.023650 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3c64b1d5-148a-4149-8e6c-6f4a28cb1b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:31.049570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3815ad0-a918-45bb-b59d-674300171ef1 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:31.084220 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29d1bb-fbc5-4af7-a9fc-c1f0d51ed33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:31.084987 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29d1bb-fbc5-4af7-a9fc-c1f0d51ed33f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:31.085367 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29d1bb-fbc5-4af7-a9fc-c1f0d51ed33f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:31.085743 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29d1bb-fbc5-4af7-a9fc-c1f0d51ed33f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:31.087438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 188/386] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:31.088324 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64b1d5-148a-4149-8e6c-6f4a28cb1b63 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:31.105286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa29d1bb-fbc5-4af7-a9fc-c1f0d51ed33f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:31.106727 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa29d1bb-fbc5-4af7-a9fc-c1f0d51ed33f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:31.121433 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:31.131183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-fa29d1bb-fbc5-4af7-a9fc-c1f0d51ed33f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:31.131183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:31.131183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:31.131183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:31.131183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:31.131183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:31.131183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:31.131183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:31.131183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:31.131183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:31.131183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:31.131183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:31.131183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:31.131183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:31.131183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:31.131183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:31.132120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:31.132120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:31.132120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:31.132120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:31.132120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:31.132120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:31.132120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:31.132120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:31.132120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:31.132120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:31.132120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:31.132120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:31.132120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:31.132120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:31.132120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:31.132120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:31.132120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:31.132986 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:31.132986 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:31.132986 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:31.133546 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 199/387] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:31.134306 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29d1bb-fbc5-4af7-a9fc-c1f0d51ed33f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:31.194468 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64b1d5-148a-4149-8e6c-6f4a28cb1b63 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:31.203757 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:31.239027 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa29d1bb-fbc5-4af7-a9fc-c1f0d51ed33f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:31.243557 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a38b8-7167-4bfc-afe3-8fa70d1cd6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:31.244162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a38b8-7167-4bfc-afe3-8fa70d1cd6bf None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:31.244439 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a38b8-7167-4bfc-afe3-8fa70d1cd6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:31.244708 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a38b8-7167-4bfc-afe3-8fa70d1cd6bf None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:31.245367 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a38b8-7167-4bfc-afe3-8fa70d1cd6bf None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:31.245666 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a38b8-7167-4bfc-afe3-8fa70d1cd6bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:31.247484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a38b8-7167-4bfc-afe3-8fa70d1cd6bf None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:31.258271 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 189/388] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Oct 17 02:59:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:31.259097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a38b8-7167-4bfc-afe3-8fa70d1cd6bf None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:31.283419 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3d8bd-ed94-4ab1-a169-ed2b9452cfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:31.284583 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3d8bd-ed94-4ab1-a169-ed2b9452cfb6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:31.284871 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3d8bd-ed94-4ab1-a169-ed2b9452cfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:31.285149 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3d8bd-ed94-4ab1-a169-ed2b9452cfb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:31.295800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3d8bd-ed94-4ab1-a169-ed2b9452cfb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:31.297389 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3d8bd-ed94-4ab1-a169-ed2b9452cfb6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:31.309910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-6ea3d8bd-ed94-4ab1-a169-ed2b9452cfb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:31.309910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:31.309910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:31.309910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:31.309910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:31.309910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:31.309910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:31.309910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:31.309910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:31.309910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:31.309910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:31.309910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:31.309910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 02:59:31.309910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 02:59:31.309910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 02:59:31.311003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 02:59:31.311003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:31.311003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:31.311003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:31.311003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:31.311003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:31.311003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:31.311003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:31.311003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:31.311003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:31.311003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:31.311003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:31.311003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:31.311003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:31.311003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:31.311003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:31.311003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:31.311959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 02:59:31.311959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 02:59:31.311959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 02:59:31.311959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 02:59:31.311959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 02:59:31.311959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 02:59:31.311959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:31.311959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:31.311959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 200/389] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:31.312822 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3d8bd-ed94-4ab1-a169-ed2b9452cfb6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:31.372741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a38b8-7167-4bfc-afe3-8fa70d1cd6bf None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:31.374726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:31.407656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d8014-c84e-43e4-8219-ed470e28bc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:31.408284 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d8014-c84e-43e4-8219-ed470e28bc6b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:31.408572 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d8014-c84e-43e4-8219-ed470e28bc6b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:31.408845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d8014-c84e-43e4-8219-ed470e28bc6b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:31.415728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3d8bd-ed94-4ab1-a169-ed2b9452cfb6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:31.416321 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d8014-c84e-43e4-8219-ed470e28bc6b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:31.417793 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d8014-c84e-43e4-8219-ed470e28bc6b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:31.429311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-0b5d8014-c84e-43e4-8219-ed470e28bc6b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 02:59:31.429311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:31.429311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:31.429311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:31.429311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:31.429311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:31.429311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:31.429311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:31.429311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:31.429311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:31.429311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:31.429311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:31.429311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 02:59:31.429311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 02:59:31.429311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 02:59:31.429311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:31.430176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:31.431093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:31.431093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:31.431093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 02:59:31.431093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 02:59:31.431093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 02:59:31.431093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 02:59:31.431093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 02:59:31.431093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 02:59:31.431093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 02:59:31.431093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:31.431093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 190/390] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:31.431668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d8014-c84e-43e4-8219-ed470e28bc6b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:31.454082 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b3815-3309-4c2d-9583-5464ad7b0dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:31.454627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b3815-3309-4c2d-9583-5464ad7b0dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:31.454992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b3815-3309-4c2d-9583-5464ad7b0dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:31.455266 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b3815-3309-4c2d-9583-5464ad7b0dc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:31.455901 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b3815-3309-4c2d-9583-5464ad7b0dc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:31.456198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b3815-3309-4c2d-9583-5464ad7b0dc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:31.458172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b3815-3309-4c2d-9583-5464ad7b0dc1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:31.459371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:31.470086 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 201/391] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:31.470916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b3815-3309-4c2d-9583-5464ad7b0dc1 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:31.530971 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d8014-c84e-43e4-8219-ed470e28bc6b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:31.563501 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a8960-0d0c-49f4-8a6e-2f4d0394ecc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:31.564122 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a8960-0d0c-49f4-8a6e-2f4d0394ecc5 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:31.564399 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a8960-0d0c-49f4-8a6e-2f4d0394ecc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:31.564667 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a8960-0d0c-49f4-8a6e-2f4d0394ecc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:31.571421 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b3815-3309-4c2d-9583-5464ad7b0dc1 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:31.575350 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a8960-0d0c-49f4-8a6e-2f4d0394ecc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:31.576801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a8960-0d0c-49f4-8a6e-2f4d0394ecc5 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:31.590838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-911a8960-0d0c-49f4-8a6e-2f4d0394ecc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:31.590838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:31.590838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:31.590838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:31.590838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:31.590838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:31.590838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:31.590838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:31.590838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:31.590838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:31.590838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:31.590838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:31.590838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:31.590838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:31.590838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:31.590838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:31.591931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:31.591931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:31.591931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:31.591931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:31.591931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:31.591931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:31.591931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:31.591931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:31.591931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:31.591931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:31.591931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:31.591931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:31.591931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:31.591931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:31.591931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:31.591931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:31.591931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:31.592783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:31.592783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:31.592783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:31.592783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 191/392] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:31.593687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:31.595699 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a8960-0d0c-49f4-8a6e-2f4d0394ecc5 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:31.608547 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610f5c0-2730-46a0-be98-f66c3991e603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:31.609161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610f5c0-2730-46a0-be98-f66c3991e603 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:31.609747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610f5c0-2730-46a0-be98-f66c3991e603 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:31.610025 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610f5c0-2730-46a0-be98-f66c3991e603 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:31.610718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610f5c0-2730-46a0-be98-f66c3991e603 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:31.611054 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610f5c0-2730-46a0-be98-f66c3991e603 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:31.612489 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5610f5c0-2730-46a0-be98-f66c3991e603 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:31.630269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 202/393] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 02:59:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:31.631079 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610f5c0-2730-46a0-be98-f66c3991e603 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:31.677484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:31.697111 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a8960-0d0c-49f4-8a6e-2f4d0394ecc5 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:31.729465 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610f5c0-2730-46a0-be98-f66c3991e603 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:31.730921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e09d7-50d3-4337-8b99-f3052f32ec64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:31.731550 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e09d7-50d3-4337-8b99-f3052f32ec64 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:31.731829 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e09d7-50d3-4337-8b99-f3052f32ec64 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:31.732104 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e09d7-50d3-4337-8b99-f3052f32ec64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:31.742354 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e09d7-50d3-4337-8b99-f3052f32ec64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:31.743856 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e09d7-50d3-4337-8b99-f3052f32ec64 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:31.758325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-0a7e09d7-50d3-4337-8b99-f3052f32ec64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:31.758325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:31.758325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:31.758325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:31.758325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:31.758325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:31.758325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:31.758325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:31.758325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:31.758325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:31.758325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:31.758325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:31.758325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:31.758325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:31.758325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:31.758325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:31.759258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:31.759258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:31.759258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:31.759258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:31.759258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:31.759258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:31.759258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:31.759258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:31.759258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:31.759258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:31.759258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:31.759258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:31.759258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:31.759258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:31.759258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:31.759258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:31.759258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:31.760117 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:31.760117 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:31.760117 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:31.760117 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 192/394] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:31.760330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e09d7-50d3-4337-8b99-f3052f32ec64 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:31.765418 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535e31e-991b-467b-a331-e9ae882de107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:31.766023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535e31e-991b-467b-a331-e9ae882de107 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:31.766300 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535e31e-991b-467b-a331-e9ae882de107 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:31.766569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535e31e-991b-467b-a331-e9ae882de107 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:31.767328 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535e31e-991b-467b-a331-e9ae882de107 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:31.767631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535e31e-991b-467b-a331-e9ae882de107 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:31.769036 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535e31e-991b-467b-a331-e9ae882de107 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:31.805578 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 203/395] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Oct 17 02:59:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:31.806407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535e31e-991b-467b-a331-e9ae882de107 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:31.844671 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:31.866094 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e09d7-50d3-4337-8b99-f3052f32ec64 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:31.895439 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:31.900869 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57be11-858c-437b-9196-8d86789c69cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:31.901684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57be11-858c-437b-9196-8d86789c69cb None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:31.902098 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57be11-858c-437b-9196-8d86789c69cb None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:31.902395 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57be11-858c-437b-9196-8d86789c69cb None admin] PATH_INFO: `/v3/users/glance` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:31.904470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535e31e-991b-467b-a331-e9ae882de107 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:31.909616 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57be11-858c-437b-9196-8d86789c69cb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:31.911750 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57be11-858c-437b-9196-8d86789c69cb None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:31.922396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-3e57be11-858c-437b-9196-8d86789c69cb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 17 02:59:31.922396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:31.922396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:31.922396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:31.922396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:31.922396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:31.922396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:31.922396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:31.922396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:31.922396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:31.922396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:31.922396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:31.922396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 02:59:31.922396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 02:59:31.922396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 02:59:31.922396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:31.923474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:31.924371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:31.924371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:31.924371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 02:59:31.924371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 02:59:31.924371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 02:59:31.924371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 02:59:31.924371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 02:59:31.924371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 02:59:31.924371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 17 02:59:31.924371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:31.924371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 193/396] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:31.924371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57be11-858c-437b-9196-8d86789c69cb None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:31.943065 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205c7fa-7f89-4ddb-a96c-d041ebc8ae77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:31.943674 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205c7fa-7f89-4ddb-a96c-d041ebc8ae77 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:31.943957 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205c7fa-7f89-4ddb-a96c-d041ebc8ae77 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:31.944231 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205c7fa-7f89-4ddb-a96c-d041ebc8ae77 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:31.944872 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205c7fa-7f89-4ddb-a96c-d041ebc8ae77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:31.945181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205c7fa-7f89-4ddb-a96c-d041ebc8ae77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:31.946555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205c7fa-7f89-4ddb-a96c-d041ebc8ae77 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:31.957405 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 204/397] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:31.958179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205c7fa-7f89-4ddb-a96c-d041ebc8ae77 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:32.026380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57be11-858c-437b-9196-8d86789c69cb None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:32.052720 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:32.056159 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205c7fa-7f89-4ddb-a96c-d041ebc8ae77 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:32.060090 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fde393-e237-4263-a432-c4310ff0924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:32.060749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fde393-e237-4263-a432-c4310ff0924f None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:32.061060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fde393-e237-4263-a432-c4310ff0924f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:32.061339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fde393-e237-4263-a432-c4310ff0924f None admin] PATH_INFO: `/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/01637fc07ba846aebcb42325bed46216/roles/78dc08989b32455389b21096f5d57e17` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:32.076304 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fde393-e237-4263-a432-c4310ff0924f None admin] RBAC: Authorizing `identity:create_grant(project_id=89b0ee95955f465eb9ee5b8ae337d9c2, user_id=01637fc07ba846aebcb42325bed46216, role_id=78dc08989b32455389b21096f5d57e17)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:32.077763 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fde393-e237-4263-a432-c4310ff0924f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:32.090110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 194/398] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 02:59:31 2022] PUT /identity/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/01637fc07ba846aebcb42325bed46216/roles/78dc08989b32455389b21096f5d57e17 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:59:32.091040 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fde393-e237-4263-a432-c4310ff0924f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:32.091311 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548ae63-61c8-4cf1-81e2-3d080f865df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:32.092176 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548ae63-61c8-4cf1-81e2-3d080f865df7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:32.092627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548ae63-61c8-4cf1-81e2-3d080f865df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:32.093055 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548ae63-61c8-4cf1-81e2-3d080f865df7 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:32.094009 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548ae63-61c8-4cf1-81e2-3d080f865df7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:32.094467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548ae63-61c8-4cf1-81e2-3d080f865df7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:32.095928 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2548ae63-61c8-4cf1-81e2-3d080f865df7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:32.116101 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 205/399] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:32.117417 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548ae63-61c8-4cf1-81e2-3d080f865df7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:32.147962 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:32.211637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fde393-e237-4263-a432-c4310ff0924f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:32.240740 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548ae63-61c8-4cf1-81e2-3d080f865df7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:32.255621 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598948f4-ae28-41bb-9cdc-c6404b5d3cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:32.256322 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598948f4-ae28-41bb-9cdc-c6404b5d3cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:32.256596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598948f4-ae28-41bb-9cdc-c6404b5d3cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:32.256866 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598948f4-ae28-41bb-9cdc-c6404b5d3cc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:32.270109 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598948f4-ae28-41bb-9cdc-c6404b5d3cc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:32.271710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598948f4-ae28-41bb-9cdc-c6404b5d3cc4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:32.284723 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd96ec-89f1-4f41-b853-0e4e3b078e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:32.285369 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd96ec-89f1-4f41-b853-0e4e3b078e73 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:32.285654 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd96ec-89f1-4f41-b853-0e4e3b078e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:32.285930 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd96ec-89f1-4f41-b853-0e4e3b078e73 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:32.286664 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd96ec-89f1-4f41-b853-0e4e3b078e73 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:32.287001 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd96ec-89f1-4f41-b853-0e4e3b078e73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:32.288439 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd96ec-89f1-4f41-b853-0e4e3b078e73 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:32.288598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-598948f4-ae28-41bb-9cdc-c6404b5d3cc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:32.288598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:32.288598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:32.288598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:32.288598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:32.288598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:32.288598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:32.288598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:32.288598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:32.288598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:32.288598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:32.288598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:32.288598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:32.288598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:32.288598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:32.288598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:32.289526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:32.289526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:32.289526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:32.289526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:32.289526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:32.289526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:32.289526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:32.289526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:32.289526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:32.289526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:32.289526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:32.289526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:32.289526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:32.289526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:32.289526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:32.289526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:32.289526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:32.290530 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:32.290530 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:32.290530 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:32.290530 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 195/400] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:32.290797 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598948f4-ae28-41bb-9cdc-c6404b5d3cc4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:32.309432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 206/401] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Oct 17 02:59:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:32.310055 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd96ec-89f1-4f41-b853-0e4e3b078e73 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:32.329420 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:32.399903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:32.406478 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598948f4-ae28-41bb-9cdc-c6404b5d3cc4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:32.420158 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd96ec-89f1-4f41-b853-0e4e3b078e73 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:32.445558 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb5ecd-79dd-4feb-86f7-3ab65c00ea3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:32.446193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb5ecd-79dd-4feb-86f7-3ab65c00ea3b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:32.446477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb5ecd-79dd-4feb-86f7-3ab65c00ea3b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:32.446752 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb5ecd-79dd-4feb-86f7-3ab65c00ea3b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:32.454562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb5ecd-79dd-4feb-86f7-3ab65c00ea3b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:32.454696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b58a1-c1e9-40a4-aefc-2fef3292d8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:32.455330 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b58a1-c1e9-40a4-aefc-2fef3292d8d9 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:32.455612 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b58a1-c1e9-40a4-aefc-2fef3292d8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:32.455891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b58a1-c1e9-40a4-aefc-2fef3292d8d9 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:32.456060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb5ecd-79dd-4feb-86f7-3ab65c00ea3b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:32.456545 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b58a1-c1e9-40a4-aefc-2fef3292d8d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:32.458782 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b58a1-c1e9-40a4-aefc-2fef3292d8d9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:32.468194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-54bb5ecd-79dd-4feb-86f7-3ab65c00ea3b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 02:59:32.468194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:32.468194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:32.468194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:32.468194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:32.468194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:32.468194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:32.468194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:32.468194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:32.468194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:32.468194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:32.468194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:32.468194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 02:59:32.468194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 02:59:32.468194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 02:59:32.468194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:32.469064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:32.469939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:32.469939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:32.469939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 02:59:32.469939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 02:59:32.469939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 02:59:32.469939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 02:59:32.469939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 02:59:32.469939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 02:59:32.469939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 02:59:32.469939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:32.469939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 196/402] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Oct 17 02:59:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:32.471109 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb5ecd-79dd-4feb-86f7-3ab65c00ea3b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:32.498896 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 207/403] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 02:59:32 2022] POST /identity/v3/users => generated 247 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:59:32.498896 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b58a1-c1e9-40a4-aefc-2fef3292d8d9 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:32.576371 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:32.618857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b58a1-c1e9-40a4-aefc-2fef3292d8d9 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:32.621353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb5ecd-79dd-4feb-86f7-3ab65c00ea3b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:32.660922 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76155b5-7998-4767-82a7-610e0d5bcc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:32.661755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76155b5-7998-4767-82a7-610e0d5bcc7b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:32.662136 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76155b5-7998-4767-82a7-610e0d5bcc7b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:32.662511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76155b5-7998-4767-82a7-610e0d5bcc7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:32.668306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658e402-14fe-45fc-89ac-69ad1f98ac58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:32.668988 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658e402-14fe-45fc-89ac-69ad1f98ac58 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:32.669270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658e402-14fe-45fc-89ac-69ad1f98ac58 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:32.669545 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658e402-14fe-45fc-89ac-69ad1f98ac58 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:32.670207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658e402-14fe-45fc-89ac-69ad1f98ac58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:32.670514 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658e402-14fe-45fc-89ac-69ad1f98ac58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:32.672025 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658e402-14fe-45fc-89ac-69ad1f98ac58 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:32.677329 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76155b5-7998-4767-82a7-610e0d5bcc7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:32.679443 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76155b5-7998-4767-82a7-610e0d5bcc7b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:32.688478 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 197/404] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:32.689127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658e402-14fe-45fc-89ac-69ad1f98ac58 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:32.695504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-b76155b5-7998-4767-82a7-610e0d5bcc7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:32.695504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:32.695504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:32.695504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:32.695504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:32.695504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:32.695504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:32.695504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:32.695504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:32.695504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:32.695504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:32.695504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:32.695504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:32.695504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:32.695504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:32.695504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:32.697582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:32.697582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:32.697582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:32.697582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:32.697582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:32.697582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:32.697582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:32.697582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:32.697582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:32.697582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:32.697582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:32.697582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:32.697582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:32.697582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:32.697582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:32.697582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:32.697582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:32.699789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:32.699789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:32.699789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:32.699789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 208/405] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:32.699789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76155b5-7998-4767-82a7-610e0d5bcc7b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:32.713567 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:32.818308 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76155b5-7998-4767-82a7-610e0d5bcc7b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:32.820908 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658e402-14fe-45fc-89ac-69ad1f98ac58 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:32.843492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:32.856751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e644f4-0bcd-4cd6-9571-647690f9d661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:32.857678 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e644f4-0bcd-4cd6-9571-647690f9d661 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:32.858097 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e644f4-0bcd-4cd6-9571-647690f9d661 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:32.858508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e644f4-0bcd-4cd6-9571-647690f9d661 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:32.861315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c491b15-3ce8-401e-9676-795691748b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:32.862053 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c491b15-3ce8-401e-9676-795691748b2d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:32.862434 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c491b15-3ce8-401e-9676-795691748b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:32.862808 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c491b15-3ce8-401e-9676-795691748b2d None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:32.864086 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c491b15-3ce8-401e-9676-795691748b2d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:32.864572 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c491b15-3ce8-401e-9676-795691748b2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:32.866327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c491b15-3ce8-401e-9676-795691748b2d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:32.867166 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e644f4-0bcd-4cd6-9571-647690f9d661 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:32.868741 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e644f4-0bcd-4cd6-9571-647690f9d661 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:32.882563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-71e644f4-0bcd-4cd6-9571-647690f9d661 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:32.882563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:32.882563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:32.882563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:32.882563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:32.882563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:32.882563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:32.882563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:32.882563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:32.882563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:32.882563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:32.882563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:32.882563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 02:59:32.882563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 02:59:32.882563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 02:59:32.883652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 02:59:32.883652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:32.883652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:32.883652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:32.883652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:32.883652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:32.883652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:32.883652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:32.883652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:32.883652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:32.883652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:32.883652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:32.883652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:32.883652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:32.883652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:32.883652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:32.883652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:32.884553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 02:59:32.884553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 02:59:32.884553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 02:59:32.884553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 02:59:32.884553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 02:59:32.884553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 02:59:32.884553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:32.884553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:32.890904 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 198/407] 10.222.0.27 () {56 vars in 1139 bytes} [Mon Oct 17 02:59:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:32.890904 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c491b15-3ce8-401e-9676-795691748b2d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:32.891454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 209/407] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:32.893022 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e644f4-0bcd-4cd6-9571-647690f9d661 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:32.969250 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:33.000296 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e644f4-0bcd-4cd6-9571-647690f9d661 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:33.006162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c491b15-3ce8-401e-9676-795691748b2d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:33.037947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb24900-8420-43bc-a2d5-dfa1968509ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:33.038790 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb24900-8420-43bc-a2d5-dfa1968509ab None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:33.039232 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb24900-8420-43bc-a2d5-dfa1968509ab None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:33.039642 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb24900-8420-43bc-a2d5-dfa1968509ab None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:33.040469 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb24900-8420-43bc-a2d5-dfa1968509ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:33.040936 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb24900-8420-43bc-a2d5-dfa1968509ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:33.042622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb24900-8420-43bc-a2d5-dfa1968509ab None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:33.044380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb96764-0bec-4a1e-91be-5eb81ef11ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:33.045090 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb96764-0bec-4a1e-91be-5eb81ef11ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:33.045466 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb96764-0bec-4a1e-91be-5eb81ef11ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:33.046025 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb96764-0bec-4a1e-91be-5eb81ef11ab7 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:33.046577 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb96764-0bec-4a1e-91be-5eb81ef11ab7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:33.047030 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb96764-0bec-4a1e-91be-5eb81ef11ab7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:33.048508 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb96764-0bec-4a1e-91be-5eb81ef11ab7 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:33.062452 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 210/408] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 02:59:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:33.063643 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb24900-8420-43bc-a2d5-dfa1968509ab None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:33.066534 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 199/409] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:33.068044 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb96764-0bec-4a1e-91be-5eb81ef11ab7 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:33.118281 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:33.172893 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb96764-0bec-4a1e-91be-5eb81ef11ab7 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:33.175307 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb24900-8420-43bc-a2d5-dfa1968509ab None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:33.205369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:33.210686 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20caa9d-be86-42c7-ae9f-eb38f847c1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:33.211456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20caa9d-be86-42c7-ae9f-eb38f847c1d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:33.211805 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20caa9d-be86-42c7-ae9f-eb38f847c1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:33.212146 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20caa9d-be86-42c7-ae9f-eb38f847c1d2 None admin] PATH_INFO: `/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/67abf21b99d349dfa7d88a0e18be1fed/roles/78dc08989b32455389b21096f5d57e17` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:33.220946 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1588ac0-b47e-465d-996b-fa41561fc36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:33.221725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1588ac0-b47e-465d-996b-fa41561fc36d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:33.222077 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1588ac0-b47e-465d-996b-fa41561fc36d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:33.222421 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1588ac0-b47e-465d-996b-fa41561fc36d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:33.234077 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1588ac0-b47e-465d-996b-fa41561fc36d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:33.235620 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20caa9d-be86-42c7-ae9f-eb38f847c1d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=89b0ee95955f465eb9ee5b8ae337d9c2, user_id=67abf21b99d349dfa7d88a0e18be1fed, role_id=78dc08989b32455389b21096f5d57e17)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:33.235933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1588ac0-b47e-465d-996b-fa41561fc36d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:33.237904 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20caa9d-be86-42c7-ae9f-eb38f847c1d2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:33.258740 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 200/410] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 02:59:33 2022] PUT /identity/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/67abf21b99d349dfa7d88a0e18be1fed/roles/78dc08989b32455389b21096f5d57e17 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:59:33.260139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20caa9d-be86-42c7-ae9f-eb38f847c1d2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:33.266687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-b1588ac0-b47e-465d-996b-fa41561fc36d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:33.266687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:33.266687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:33.266687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:33.266687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:33.266687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:33.266687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:33.266687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:33.266687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:33.266687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:33.266687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:33.266687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:33.266687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:33.266687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:33.266687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:33.266687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:33.267696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:33.267696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:33.267696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:33.267696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:33.267696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:33.267696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:33.267696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:33.267696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:33.267696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:33.267696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:33.267696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:33.267696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:33.267696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:33.267696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:33.267696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:33.267696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:33.267696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:33.268564 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:33.268564 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:33.268564 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:33.268564 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 211/411] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:33.274799 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1588ac0-b47e-465d-996b-fa41561fc36d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:33.441465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20caa9d-be86-42c7-ae9f-eb38f847c1d2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:33.490787 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:33.497715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1588ac0-b47e-465d-996b-fa41561fc36d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:33.507483 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263f63d-c0a2-4d60-94fd-63f19a2a00b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:33.508408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263f63d-c0a2-4d60-94fd-63f19a2a00b4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:33.508840 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263f63d-c0a2-4d60-94fd-63f19a2a00b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:33.509304 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263f63d-c0a2-4d60-94fd-63f19a2a00b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:33.517291 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263f63d-c0a2-4d60-94fd-63f19a2a00b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:33.519267 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263f63d-c0a2-4d60-94fd-63f19a2a00b4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:33.532777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-8263f63d-c0a2-4d60-94fd-63f19a2a00b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:33.532777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:33.532777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:33.532777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:33.532777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:33.532777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:33.532777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:33.532777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:33.532777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:33.532777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:33.532777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:33.532777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:33.532777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 02:59:33.532777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 02:59:33.532777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 02:59:33.533661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 02:59:33.533661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:33.533661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:33.533661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:33.533661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:33.533661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:33.533661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:33.533661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:33.533661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:33.533661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:33.533661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:33.533661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:33.533661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:33.533661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:33.533661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:33.533661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:33.533661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:33.534545 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 02:59:33.534545 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 02:59:33.534545 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 02:59:33.534545 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 02:59:33.534545 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 02:59:33.534545 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 02:59:33.534545 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:33.534545 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:33.536827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 201/412] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:33.542902 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263f63d-c0a2-4d60-94fd-63f19a2a00b4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:33.543795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4dd621-aa6c-42c9-8c05-861c4ec089b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:33.544981 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4dd621-aa6c-42c9-8c05-861c4ec089b5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:33.547106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4dd621-aa6c-42c9-8c05-861c4ec089b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:33.547395 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4dd621-aa6c-42c9-8c05-861c4ec089b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:33.548070 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4dd621-aa6c-42c9-8c05-861c4ec089b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:33.548378 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4dd621-aa6c-42c9-8c05-861c4ec089b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:33.549761 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4dd621-aa6c-42c9-8c05-861c4ec089b5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:33.566317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 212/413] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:33.573447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4dd621-aa6c-42c9-8c05-861c4ec089b5 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:33.602011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:33.702931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263f63d-c0a2-4d60-94fd-63f19a2a00b4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:33.731594 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4dd621-aa6c-42c9-8c05-861c4ec089b5 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:33.768671 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13996cbb-35fe-4d05-8352-c83f28d2bae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:33.769571 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13996cbb-35fe-4d05-8352-c83f28d2bae8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:33.770545 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13996cbb-35fe-4d05-8352-c83f28d2bae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:33.771032 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13996cbb-35fe-4d05-8352-c83f28d2bae8 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:33.771914 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13996cbb-35fe-4d05-8352-c83f28d2bae8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:33.772401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13996cbb-35fe-4d05-8352-c83f28d2bae8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:33.774223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13996cbb-35fe-4d05-8352-c83f28d2bae8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:33.797760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748837c-d7f6-4eeb-af37-12f7602bf945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:33.798626 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748837c-d7f6-4eeb-af37-12f7602bf945 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:33.799088 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748837c-d7f6-4eeb-af37-12f7602bf945 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:33.799503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748837c-d7f6-4eeb-af37-12f7602bf945 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:33.806553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748837c-d7f6-4eeb-af37-12f7602bf945 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:33.808221 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748837c-d7f6-4eeb-af37-12f7602bf945 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:33.809783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 202/414] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 02:59:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:33.810833 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13996cbb-35fe-4d05-8352-c83f28d2bae8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:33.820571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-8748837c-d7f6-4eeb-af37-12f7602bf945 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:33.820571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:33.820571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:33.820571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:33.820571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:33.820571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:33.820571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:33.820571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:33.820571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:33.820571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:33.820571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:33.820571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:33.820571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 02:59:33.820571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 02:59:33.820571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 02:59:33.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 02:59:33.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:33.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:33.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:33.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:33.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:33.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:33.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:33.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:33.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:33.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:33.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:33.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:33.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:33.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:33.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:33.821640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:33.822712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 02:59:33.822712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 02:59:33.822712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 02:59:33.822712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 02:59:33.822712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 02:59:33.822712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 02:59:33.822712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:33.822712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:33.824402 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 213/415] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:59:33.830927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748837c-d7f6-4eeb-af37-12f7602bf945 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:33.866212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:33.934572 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13996cbb-35fe-4d05-8352-c83f28d2bae8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:33.940153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:34.007871 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8731db-7d7a-40ea-8537-55dc15ce6a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:34.008454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8731db-7d7a-40ea-8537-55dc15ce6a25 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:34.008733 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8731db-7d7a-40ea-8537-55dc15ce6a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:34.009008 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8731db-7d7a-40ea-8537-55dc15ce6a25 None admin] PATH_INFO: `/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/a8aa29639a3143d2a270e3248d2205d1/roles/78dc08989b32455389b21096f5d57e17` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:34.038601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748837c-d7f6-4eeb-af37-12f7602bf945 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:34.045746 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8731db-7d7a-40ea-8537-55dc15ce6a25 None admin] RBAC: Authorizing `identity:create_grant(project_id=89b0ee95955f465eb9ee5b8ae337d9c2, user_id=a8aa29639a3143d2a270e3248d2205d1, role_id=78dc08989b32455389b21096f5d57e17)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:34.047462 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8731db-7d7a-40ea-8537-55dc15ce6a25 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:34.075758 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 203/416] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 02:59:33 2022] PUT /identity/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/a8aa29639a3143d2a270e3248d2205d1/roles/78dc08989b32455389b21096f5d57e17 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 02:59:34.080694 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749dd6d-e393-4803-a50f-df6db87c6097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:34.081715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749dd6d-e393-4803-a50f-df6db87c6097 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:34.082269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749dd6d-e393-4803-a50f-df6db87c6097 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:34.082799 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749dd6d-e393-4803-a50f-df6db87c6097 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:34.085168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749dd6d-e393-4803-a50f-df6db87c6097 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:34.085786 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749dd6d-e393-4803-a50f-df6db87c6097 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:34.087583 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749dd6d-e393-4803-a50f-df6db87c6097 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:34.104118 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 214/417] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 02:59:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:34.110552 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8731db-7d7a-40ea-8537-55dc15ce6a25 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:34.290646 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8731db-7d7a-40ea-8537-55dc15ce6a25 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:34.351658 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e883286-d56f-45ee-8276-e50c06f4f1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:34.352352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e883286-d56f-45ee-8276-e50c06f4f1b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:34.352635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e883286-d56f-45ee-8276-e50c06f4f1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:34.352909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e883286-d56f-45ee-8276-e50c06f4f1b4 None admin] PATH_INFO: `/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/526933e6ad004493b071f5da5caa09a3/roles/78dc08989b32455389b21096f5d57e17` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:34.370584 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e883286-d56f-45ee-8276-e50c06f4f1b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=89b0ee95955f465eb9ee5b8ae337d9c2, user_id=526933e6ad004493b071f5da5caa09a3, role_id=78dc08989b32455389b21096f5d57e17)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:34.372158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e883286-d56f-45ee-8276-e50c06f4f1b4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:34.387712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 204/418] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 02:59:34 2022] PUT /identity/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/526933e6ad004493b071f5da5caa09a3/roles/78dc08989b32455389b21096f5d57e17 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:59:34.409890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40406283-800e-44ef-8788-0881e7fd82a9 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:34.410790 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40406283-800e-44ef-8788-0881e7fd82a9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:34.411305 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40406283-800e-44ef-8788-0881e7fd82a9 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:34.414324 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 215/419] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:34.418910 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b66b96-f671-4cc4-ba20-2aa05f738c7d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:34.419508 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b66b96-f671-4cc4-ba20-2aa05f738c7d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:34.419994 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b66b96-f671-4cc4-ba20-2aa05f738c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:34.427867 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:34.442954 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e9b66b96-f671-4cc4-ba20-2aa05f738c7d None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:34.444337 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e9b66b96-f671-4cc4-ba20-2aa05f738c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:34.532918 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 205/420] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:34.847725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40406283-800e-44ef-8788-0881e7fd82a9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:34.942036 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:34.976480 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40406283-800e-44ef-8788-0881e7fd82a9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:35.020185 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90dd65-91ff-4477-9c4d-9e54e24f9423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:35.020905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90dd65-91ff-4477-9c4d-9e54e24f9423 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:35.021224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90dd65-91ff-4477-9c4d-9e54e24f9423 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:35.021498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90dd65-91ff-4477-9c4d-9e54e24f9423 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:35.043514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd90dd65-91ff-4477-9c4d-9e54e24f9423 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:35.048540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-fd90dd65-91ff-4477-9c4d-9e54e24f9423 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:35.048540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:35.048540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:35.048540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:35.048540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:35.048540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:35.048540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:35.048540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:35.048540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:35.048540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:35.048540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:35.048540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:35.048540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:35.048540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:35.048540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:35.048540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:35.049529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:35.049529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:35.049529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:35.049529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:35.049529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:35.049529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:35.049529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:35.049529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:35.049529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:35.049529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:35.049529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:35.049529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:35.049529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:35.049529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:35.049529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:35.049529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:35.050354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:35.050354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:35.050354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:35.050354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:35.050354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:35.050354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:35.050354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:35.050354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:35.050354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:35.050354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:35.050354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:35.050354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:35.050354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 216/421] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Oct 17 02:59:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:35.058737 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b66b96-f671-4cc4-ba20-2aa05f738c7d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:35.163688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992c802-92f3-448b-807c-695501ee00ee None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:35.163982 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992c802-92f3-448b-807c-695501ee00ee None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:35.164335 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992c802-92f3-448b-807c-695501ee00ee None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:35.166041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 217/422] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:35.172798 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4a401-9565-44f0-9eed-7efc35dba828 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:35.173090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4a401-9565-44f0-9eed-7efc35dba828 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:35.173367 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4a401-9565-44f0-9eed-7efc35dba828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:35.210522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-eec4a401-9565-44f0-9eed-7efc35dba828 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:35.211776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-eec4a401-9565-44f0-9eed-7efc35dba828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:35.236419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b66b96-f671-4cc4-ba20-2aa05f738c7d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:35.268416 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:35.275533 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e6ecf-70b5-49c8-8f88-c94058c28251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:35.276170 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e6ecf-70b5-49c8-8f88-c94058c28251 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:35.276453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e6ecf-70b5-49c8-8f88-c94058c28251 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:35.276729 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e6ecf-70b5-49c8-8f88-c94058c28251 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:35.277419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e6ecf-70b5-49c8-8f88-c94058c28251 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:35.280563 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e6ecf-70b5-49c8-8f88-c94058c28251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:35.282307 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e6ecf-70b5-49c8-8f88-c94058c28251 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:35.292891 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 206/423] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Oct 17 02:59:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:35.303878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e6ecf-70b5-49c8-8f88-c94058c28251 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:35.375362 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 218/424] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:59:35.490187 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e6ecf-70b5-49c8-8f88-c94058c28251 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:35.541353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69779fe9-000f-4099-8884-2d3b3ea5fbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:35.541979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69779fe9-000f-4099-8884-2d3b3ea5fbd2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:35.542257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69779fe9-000f-4099-8884-2d3b3ea5fbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:35.542529 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69779fe9-000f-4099-8884-2d3b3ea5fbd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:35.558331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69779fe9-000f-4099-8884-2d3b3ea5fbd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:35.559800 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69779fe9-000f-4099-8884-2d3b3ea5fbd2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:35.574540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-69779fe9-000f-4099-8884-2d3b3ea5fbd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:35.574540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:35.574540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:35.574540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:35.574540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:35.574540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:35.574540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:35.574540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:35.574540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:35.574540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:35.574540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:35.574540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:35.574540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:35.574540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:35.574540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:35.574540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:35.575523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:35.575523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:35.575523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:35.575523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:35.575523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:35.575523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:35.575523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:35.575523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:35.575523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:35.575523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:35.575523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:35.575523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:35.575523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:35.575523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:35.575523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:35.575523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:35.575523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:35.576375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:35.576375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:35.576375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:35.576375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 207/425] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:35.581454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4a401-9565-44f0-9eed-7efc35dba828 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:35.680738 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69779fe9-000f-4099-8884-2d3b3ea5fbd2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:35.684601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:35.757157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4a401-9565-44f0-9eed-7efc35dba828 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:35.779248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:35.835593 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32e1d5-35ad-4c57-8ab8-8260712fdf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:35.836639 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32e1d5-35ad-4c57-8ab8-8260712fdf21 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:35.837122 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32e1d5-35ad-4c57-8ab8-8260712fdf21 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:35.837605 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32e1d5-35ad-4c57-8ab8-8260712fdf21 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:35.838527 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32e1d5-35ad-4c57-8ab8-8260712fdf21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:35.839110 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32e1d5-35ad-4c57-8ab8-8260712fdf21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:35.840689 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32e1d5-35ad-4c57-8ab8-8260712fdf21 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:35.851963 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69779fe9-000f-4099-8884-2d3b3ea5fbd2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:35.855035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 219/426] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:35.856920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd743e-ed93-435d-bcc5-901f86fac3c5 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:35.857415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd743e-ed93-435d-bcc5-901f86fac3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:35.857886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dd743e-ed93-435d-bcc5-901f86fac3c5 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:35.859752 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 220/427] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:35.860671 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd743e-ed93-435d-bcc5-901f86fac3c5 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:35.895995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48848f-48e6-418a-9308-6cf6f08fd626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:35.896642 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48848f-48e6-418a-9308-6cf6f08fd626 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:35.897044 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48848f-48e6-418a-9308-6cf6f08fd626 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:35.897457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48848f-48e6-418a-9308-6cf6f08fd626 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:35.909676 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48848f-48e6-418a-9308-6cf6f08fd626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:35.916485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-cb48848f-48e6-418a-9308-6cf6f08fd626 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:35.916485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:35.916485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:35.916485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:35.916485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:35.916485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:35.916485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:35.916485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:35.916485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:35.916485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:35.916485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:35.916485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:35.916485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:35.916485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:35.916485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:35.916485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:35.917402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:35.917402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:35.917402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:35.917402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:35.917402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:35.917402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:35.917402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:35.917402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:35.917402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:35.917402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:35.917402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:35.917402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:35.917402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:35.917402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:35.917402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:35.917402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:35.918213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:35.918213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:35.918213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:35.918213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:35.918213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:35.918213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:35.918213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:35.918213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:35.918213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:35.918213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:35.918213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:35.918213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:35.918213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 208/428] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Oct 17 02:59:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:59:35.923435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d47fc-0a50-4a04-b1a6-1351d93156cc None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:35.923435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d47fc-0a50-4a04-b1a6-1351d93156cc None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:35.923435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d47fc-0a50-4a04-b1a6-1351d93156cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:35.948495 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-cf4d47fc-0a50-4a04-b1a6-1351d93156cc None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:35.949858 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cf4d47fc-0a50-4a04-b1a6-1351d93156cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:35.982186 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dd743e-ed93-435d-bcc5-901f86fac3c5 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:36.014517 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:36.027131 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e967ad-17f6-4a6a-8aec-315a3e1ce102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:36.027974 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e967ad-17f6-4a6a-8aec-315a3e1ce102 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:36.028444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e967ad-17f6-4a6a-8aec-315a3e1ce102 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:36.028856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e967ad-17f6-4a6a-8aec-315a3e1ce102 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:36.031188 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 209/429] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:36.031961 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d47fc-0a50-4a04-b1a6-1351d93156cc None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:36.038436 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e967ad-17f6-4a6a-8aec-315a3e1ce102 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:36.040117 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e967ad-17f6-4a6a-8aec-315a3e1ce102 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:36.055577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d6e967ad-17f6-4a6a-8aec-315a3e1ce102 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 02:59:36.055577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:36.055577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:36.055577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:36.055577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:36.055577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:36.055577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:36.055577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:36.055577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:36.055577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:36.055577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:36.055577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:36.055577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 02:59:36.055577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 02:59:36.055577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 02:59:36.055577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:36.056522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:36.057482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:36.057482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:36.057482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 02:59:36.057482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 02:59:36.057482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 02:59:36.057482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 02:59:36.057482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 02:59:36.057482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 02:59:36.057482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 02:59:36.057482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:36.060693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 221/430] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Oct 17 02:59:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:36.066918 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e967ad-17f6-4a6a-8aec-315a3e1ce102 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:36.098196 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:36.170065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d47fc-0a50-4a04-b1a6-1351d93156cc None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:36.221544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9201e-12a8-4ccb-8396-038aaef82782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:36.222496 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9201e-12a8-4ccb-8396-038aaef82782 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:36.222987 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9201e-12a8-4ccb-8396-038aaef82782 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:36.223533 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9201e-12a8-4ccb-8396-038aaef82782 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:36.224351 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f9201e-12a8-4ccb-8396-038aaef82782 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:36.224797 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f9201e-12a8-4ccb-8396-038aaef82782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:36.226519 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f9201e-12a8-4ccb-8396-038aaef82782 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:36.235499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 210/431] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Oct 17 02:59:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:36.244839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9201e-12a8-4ccb-8396-038aaef82782 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:36.260689 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e967ad-17f6-4a6a-8aec-315a3e1ce102 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:36.331174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ddb786-daaa-49b7-a940-1d33c707d645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:36.331796 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddb786-daaa-49b7-a940-1d33c707d645 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:36.332081 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddb786-daaa-49b7-a940-1d33c707d645 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:36.332492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddb786-daaa-49b7-a940-1d33c707d645 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:36.333474 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ddb786-daaa-49b7-a940-1d33c707d645 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:36.333789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ddb786-daaa-49b7-a940-1d33c707d645 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:36.339410 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ddb786-daaa-49b7-a940-1d33c707d645 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:36.369443 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 222/432] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Oct 17 02:59:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:36.375727 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ddb786-daaa-49b7-a940-1d33c707d645 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:36.381403 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:36.390807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9201e-12a8-4ccb-8396-038aaef82782 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:36.428232 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96452b07-6b29-4ea0-97a5-1c6e0327fe6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:36.428868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96452b07-6b29-4ea0-97a5-1c6e0327fe6b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:36.429155 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96452b07-6b29-4ea0-97a5-1c6e0327fe6b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:36.429437 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96452b07-6b29-4ea0-97a5-1c6e0327fe6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:36.432921 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:36.443362 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96452b07-6b29-4ea0-97a5-1c6e0327fe6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:36.445030 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96452b07-6b29-4ea0-97a5-1c6e0327fe6b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:36.460542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-96452b07-6b29-4ea0-97a5-1c6e0327fe6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:36.460542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:36.460542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:36.460542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:36.460542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:36.460542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:36.460542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:36.460542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:36.460542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:36.460542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:36.460542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:36.460542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:36.460542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:36.460542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:36.460542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:36.460542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:36.461491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:36.461491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:36.461491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:36.461491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:36.461491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:36.461491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:36.461491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:36.461491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:36.461491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:36.461491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:36.461491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:36.461491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:36.461491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:36.461491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:36.461491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:36.461491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:36.461491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:36.462331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:36.462331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:36.462331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:36.462331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 211/433] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:59:36.463006 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96452b07-6b29-4ea0-97a5-1c6e0327fe6b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:36.490723 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ddb786-daaa-49b7-a940-1d33c707d645 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:36.527183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339a29d-2724-4ef5-926d-566e147b8a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:36.527811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339a29d-2724-4ef5-926d-566e147b8a0c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:36.528099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339a29d-2724-4ef5-926d-566e147b8a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:36.528376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339a29d-2724-4ef5-926d-566e147b8a0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:36.575548 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96452b07-6b29-4ea0-97a5-1c6e0327fe6b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:36.601177 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339a29d-2724-4ef5-926d-566e147b8a0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:36.602667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339a29d-2724-4ef5-926d-566e147b8a0c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:36.612288 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b98876-29bd-4e01-9e60-9e79df4823bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:36.613019 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b98876-29bd-4e01-9e60-9e79df4823bc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:36.613309 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b98876-29bd-4e01-9e60-9e79df4823bc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:36.613649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b98876-29bd-4e01-9e60-9e79df4823bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:36.625094 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b98876-29bd-4e01-9e60-9e79df4823bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:36.631921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-95b98876-29bd-4e01-9e60-9e79df4823bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:36.631921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:36.631921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:36.631921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:36.631921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:36.631921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:36.631921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:36.631921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:36.631921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:36.631921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:36.631921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:36.631921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:36.631921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:36.631921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:36.631921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:36.631921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:36.633972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:36.633972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:36.633972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:36.633972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:36.633972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:36.633972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:36.633972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:36.633972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:36.633972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:36.633972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:36.633972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:36.633972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:36.633972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:36.633972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:36.633972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:36.633972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:36.635864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:36.635864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:36.635864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:36.635864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:36.635864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:36.635864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:36.635864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:36.635864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:36.635864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:36.635864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:36.635864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:36.635864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:36.635864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 212/434] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Oct 17 02:59:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:36.635864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b98876-29bd-4e01-9e60-9e79df4823bc None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:36.649521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:36.654369 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:36.714317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 223/435] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:59:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 02:59:36.719621 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339a29d-2724-4ef5-926d-566e147b8a0c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:36.750544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b98876-29bd-4e01-9e60-9e79df4823bc None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:36.790295 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e212df-9ff4-4982-a09a-2a14ea403e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:36.791028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e212df-9ff4-4982-a09a-2a14ea403e93 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:36.791480 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e212df-9ff4-4982-a09a-2a14ea403e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:36.791829 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e212df-9ff4-4982-a09a-2a14ea403e93 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:36.792654 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e212df-9ff4-4982-a09a-2a14ea403e93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:36.793035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e212df-9ff4-4982-a09a-2a14ea403e93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:36.794581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e212df-9ff4-4982-a09a-2a14ea403e93 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:36.813822 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 213/436] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:36.814805 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e212df-9ff4-4982-a09a-2a14ea403e93 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:36.828416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339a29d-2724-4ef5-926d-566e147b8a0c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:36.865319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d6057-20b9-4341-868b-15bdacfc4040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:36.865319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d6057-20b9-4341-868b-15bdacfc4040 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:36.865319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d6057-20b9-4341-868b-15bdacfc4040 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:36.866580 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d6057-20b9-4341-868b-15bdacfc4040 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:36.866741 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d6057-20b9-4341-868b-15bdacfc4040 None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:36.867043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d6057-20b9-4341-868b-15bdacfc4040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:36.868506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d6057-20b9-4341-868b-15bdacfc4040 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:36.879219 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 224/437] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Oct 17 02:59:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:36.880310 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d6057-20b9-4341-868b-15bdacfc4040 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:36.894364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:36.897833 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:36.931479 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e212df-9ff4-4982-a09a-2a14ea403e93 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:36.970028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706167e-0f34-4de8-bb3d-0eb8c9061815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:36.970674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706167e-0f34-4de8-bb3d-0eb8c9061815 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:36.970986 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706167e-0f34-4de8-bb3d-0eb8c9061815 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:36.971262 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706167e-0f34-4de8-bb3d-0eb8c9061815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:36.981487 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706167e-0f34-4de8-bb3d-0eb8c9061815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:36.982985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706167e-0f34-4de8-bb3d-0eb8c9061815 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:36.999054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-b706167e-0f34-4de8-bb3d-0eb8c9061815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:36.999054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:36.999054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:36.999054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:36.999054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:36.999054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:36.999054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:36.999054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:36.999054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:36.999054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:36.999054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:36.999054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:36.999054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:36.999054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:36.999054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:36.999054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:37.000078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:37.000078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:37.000078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:37.000078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:37.000078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:37.000078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:37.000078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:37.000078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:37.000078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:37.000078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:37.000078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:37.000078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:37.000078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:37.000078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:37.000078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:37.000078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:37.000078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:37.000980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:37.000980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:37.000980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:37.000980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 214/438] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:37.003692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d6057-20b9-4341-868b-15bdacfc4040 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:37.006281 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acb6f1-25e5-4052-a9a9-249a9221fb96 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:37.006575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acb6f1-25e5-4052-a9a9-249a9221fb96 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:37.006851 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acb6f1-25e5-4052-a9a9-249a9221fb96 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:37.008151 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 215/439] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:37.008859 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acb6f1-25e5-4052-a9a9-249a9221fb96 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:37.042467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b7f35-91ad-478e-80ae-62b69efebf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:37.043233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b7f35-91ad-478e-80ae-62b69efebf72 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:37.043579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b7f35-91ad-478e-80ae-62b69efebf72 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:37.043861 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b7f35-91ad-478e-80ae-62b69efebf72 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:37.051590 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b7f35-91ad-478e-80ae-62b69efebf72 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:37.053153 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b7f35-91ad-478e-80ae-62b69efebf72 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:37.064365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-c09b7f35-91ad-478e-80ae-62b69efebf72 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 02:59:37.064365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:37.064365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:37.064365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:37.064365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:37.064365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:37.064365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:37.064365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:37.064365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:37.064365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:37.064365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:37.064365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:37.064365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 02:59:37.064365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 02:59:37.064365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 02:59:37.064365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:37.065466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:37.066435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:37.066435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:37.066435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 02:59:37.066435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 02:59:37.066435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 02:59:37.066435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 02:59:37.066435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 02:59:37.066435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 02:59:37.066435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 02:59:37.066435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:37.066435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 225/440] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 02:59:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:37.067897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b7f35-91ad-478e-80ae-62b69efebf72 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:37.119733 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acb6f1-25e5-4052-a9a9-249a9221fb96 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:37.133361 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:37.142924 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:37.155713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c640d4-02a1-43ad-8e56-a0b4600e5f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:37.156346 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c640d4-02a1-43ad-8e56-a0b4600e5f9b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:37.156653 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c640d4-02a1-43ad-8e56-a0b4600e5f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:37.156925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c640d4-02a1-43ad-8e56-a0b4600e5f9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:37.167938 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c640d4-02a1-43ad-8e56-a0b4600e5f9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:37.168319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b7f35-91ad-478e-80ae-62b69efebf72 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:37.169403 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c640d4-02a1-43ad-8e56-a0b4600e5f9b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:37.184369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-e3c640d4-02a1-43ad-8e56-a0b4600e5f9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:37.184369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:37.184369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:37.184369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:37.184369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:37.184369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:37.184369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:37.184369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:37.184369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:37.184369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:37.184369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:37.184369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:37.184369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:37.184369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:37.184369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:37.184369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:37.185323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:37.185323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:37.185323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:37.185323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:37.185323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:37.185323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:37.185323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:37.185323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:37.185323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:37.185323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:37.185323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:37.185323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:37.185323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:37.185323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:37.185323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:37.185323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:37.185323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:37.186237 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:37.186237 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:37.186237 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:37.186237 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 216/441] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:37.186834 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f720b-5b39-46a2-bba4-6de6af8f533c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:37.186834 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f720b-5b39-46a2-bba4-6de6af8f533c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:37.187209 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f720b-5b39-46a2-bba4-6de6af8f533c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:37.204122 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d0ae7-b761-4a32-8f58-f7bd5571cb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:37.204732 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d0ae7-b761-4a32-8f58-f7bd5571cb90 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:37.205017 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d0ae7-b761-4a32-8f58-f7bd5571cb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:37.205291 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d0ae7-b761-4a32-8f58-f7bd5571cb90 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:37.205376 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4c0f720b-5b39-46a2-bba4-6de6af8f533c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:37.205934 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d0ae7-b761-4a32-8f58-f7bd5571cb90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:37.206241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d0ae7-b761-4a32-8f58-f7bd5571cb90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:37.206783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4c0f720b-5b39-46a2-bba4-6de6af8f533c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:37.207683 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d0ae7-b761-4a32-8f58-f7bd5571cb90 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:37.219685 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 226/442] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:37.221172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3b249-2674-47a5-a227-1c7527382eff None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:37.221526 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3b249-2674-47a5-a227-1c7527382eff None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:37.221877 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3b249-2674-47a5-a227-1c7527382eff None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:37.224113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 227/443] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:37.224927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3b249-2674-47a5-a227-1c7527382eff None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:37.277256 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 217/444] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:37.278161 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f720b-5b39-46a2-bba4-6de6af8f533c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:37.338521 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3b249-2674-47a5-a227-1c7527382eff None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:37.367301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:37.377469 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0ac96-56c4-41dd-b4b1-528ba2b62001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:37.378085 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0ac96-56c4-41dd-b4b1-528ba2b62001 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:37.378431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0ac96-56c4-41dd-b4b1-528ba2b62001 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:37.378773 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0ac96-56c4-41dd-b4b1-528ba2b62001 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:37.380417 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0ac96-56c4-41dd-b4b1-528ba2b62001 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:37.380783 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0ac96-56c4-41dd-b4b1-528ba2b62001 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:37.381751 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f720b-5b39-46a2-bba4-6de6af8f533c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:37.382340 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0ac96-56c4-41dd-b4b1-528ba2b62001 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:37.385307 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:37.399599 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 228/445] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Oct 17 02:59:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:37.404340 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0ac96-56c4-41dd-b4b1-528ba2b62001 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:37.426503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67cf25-800c-4ec5-bb6a-7fc9c4a72c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:37.427240 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67cf25-800c-4ec5-bb6a-7fc9c4a72c82 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:37.427595 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67cf25-800c-4ec5-bb6a-7fc9c4a72c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:37.427940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67cf25-800c-4ec5-bb6a-7fc9c4a72c82 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:37.428672 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67cf25-800c-4ec5-bb6a-7fc9c4a72c82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:37.429041 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67cf25-800c-4ec5-bb6a-7fc9c4a72c82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:37.430495 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67cf25-800c-4ec5-bb6a-7fc9c4a72c82 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:37.447205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 218/446] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:37.447205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5b2bc-aacf-47f5-8d24-d6afef79a80f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:37.447205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5b2bc-aacf-47f5-8d24-d6afef79a80f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:37.447205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5b2bc-aacf-47f5-8d24-d6afef79a80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:37.463020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e9b5b2bc-aacf-47f5-8d24-d6afef79a80f None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:37.464206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e9b5b2bc-aacf-47f5-8d24-d6afef79a80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:37.512995 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0ac96-56c4-41dd-b4b1-528ba2b62001 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:37.534323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 219/447] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:37.535445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5b2bc-aacf-47f5-8d24-d6afef79a80f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:37.552126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e57997-a464-4b97-b54a-fe7ebe1e21da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:37.552756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e57997-a464-4b97-b54a-fe7ebe1e21da None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:37.553033 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e57997-a464-4b97-b54a-fe7ebe1e21da None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:37.553300 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e57997-a464-4b97-b54a-fe7ebe1e21da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:37.560139 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e57997-a464-4b97-b54a-fe7ebe1e21da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:37.561575 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e57997-a464-4b97-b54a-fe7ebe1e21da None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:37.572811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d8e57997-a464-4b97-b54a-fe7ebe1e21da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:37.572811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:37.572811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:37.572811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:37.572811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:37.572811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:37.572811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:37.572811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:37.572811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:37.572811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:37.572811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:37.572811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:37.572811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 02:59:37.572811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 02:59:37.572811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 02:59:37.573677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 02:59:37.573677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:37.573677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:37.573677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:37.573677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:37.573677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:37.573677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:37.573677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:37.573677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:37.573677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:37.573677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:37.573677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:37.573677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:37.573677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:37.573677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:37.573677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:37.573677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:37.574522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 02:59:37.574522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 02:59:37.574522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 02:59:37.574522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 02:59:37.574522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 02:59:37.574522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 02:59:37.574522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:37.574522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:37.574522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 229/448] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:37.575732 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e57997-a464-4b97-b54a-fe7ebe1e21da None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:37.624084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:37.645158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:37.653801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5b2bc-aacf-47f5-8d24-d6afef79a80f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:37.683266 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e57997-a464-4b97-b54a-fe7ebe1e21da None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:37.688831 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d68469-2900-448a-af8e-83ac58703a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:37.689465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d68469-2900-448a-af8e-83ac58703a0d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:37.689745 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d68469-2900-448a-af8e-83ac58703a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:37.690357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d68469-2900-448a-af8e-83ac58703a0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:37.701068 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d68469-2900-448a-af8e-83ac58703a0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:37.702519 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d68469-2900-448a-af8e-83ac58703a0d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:37.717727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-c8d68469-2900-448a-af8e-83ac58703a0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:37.717727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:37.717727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:37.717727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:37.717727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:37.717727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:37.717727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:37.717727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:37.717727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:37.717727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:37.717727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:37.717727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:37.717727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:37.717727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:37.717727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:37.717727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:37.718657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:37.718657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:37.718657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:37.718657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:37.718657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:37.718657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:37.718657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:37.718657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:37.718657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:37.718657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:37.718657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:37.718657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:37.718657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:37.718657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:37.718657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:37.718657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:37.718657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:37.719531 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:37.719531 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:37.719531 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:37.719531 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 220/449] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:37.719754 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d68469-2900-448a-af8e-83ac58703a0d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:37.727312 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fe255-715a-49e8-85a4-01e8a6b02eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:37.727955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fe255-715a-49e8-85a4-01e8a6b02eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:37.728239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fe255-715a-49e8-85a4-01e8a6b02eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:37.728512 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fe255-715a-49e8-85a4-01e8a6b02eb2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:37.736279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867fe255-715a-49e8-85a4-01e8a6b02eb2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:37.737713 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867fe255-715a-49e8-85a4-01e8a6b02eb2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:37.749388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-867fe255-715a-49e8-85a4-01e8a6b02eb2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 02:59:37.749388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:37.749388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:37.749388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:37.749388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:37.749388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:37.749388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:37.749388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:37.749388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:37.749388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:37.749388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:37.749388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:37.749388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 02:59:37.749388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 02:59:37.749388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 02:59:37.749388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:37.750379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:37.751355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:37.751355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:37.751355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 02:59:37.751355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 02:59:37.751355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 02:59:37.751355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 02:59:37.751355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 02:59:37.751355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 02:59:37.751355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 02:59:37.751355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:37.755498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 230/450] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:37.756313 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fe255-715a-49e8-85a4-01e8a6b02eb2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:37.826983 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d68469-2900-448a-af8e-83ac58703a0d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:37.857976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fe255-715a-49e8-85a4-01e8a6b02eb2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:37.860351 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a12e7-ebac-4f3f-a358-f861ed355851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:37.860984 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a12e7-ebac-4f3f-a358-f861ed355851 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:37.861118 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:37.861262 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a12e7-ebac-4f3f-a358-f861ed355851 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:37.861931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a12e7-ebac-4f3f-a358-f861ed355851 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:37.872995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a12e7-ebac-4f3f-a358-f861ed355851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:37.878419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-9c4a12e7-ebac-4f3f-a358-f861ed355851 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:37.878419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:37.878419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:37.878419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:37.878419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:37.878419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:37.878419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:37.878419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:37.878419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:37.878419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:37.878419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:37.878419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:37.878419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:37.878419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:37.878419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:37.878419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:37.880292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:37.880292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:37.880292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:37.880292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:37.880292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:37.880292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:37.880292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:37.880292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:37.880292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:37.880292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:37.880292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:37.880292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:37.880292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:37.880292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:37.880292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:37.880292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:37.881093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:37.881093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:37.881093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:37.881093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:37.881093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:37.881093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:37.881093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:37.881093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:37.881093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:37.881093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:37.881093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:37.881093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:37.881093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 221/451] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Oct 17 02:59:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:37.881093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a12e7-ebac-4f3f-a358-f861ed355851 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:37.889370 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:37.897625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e403c2-93db-47d1-a30b-e01d1d633ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:37.898257 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e403c2-93db-47d1-a30b-e01d1d633ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:37.898545 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e403c2-93db-47d1-a30b-e01d1d633ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:37.898825 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e403c2-93db-47d1-a30b-e01d1d633ad7 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:37.899554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e403c2-93db-47d1-a30b-e01d1d633ad7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:37.899864 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e403c2-93db-47d1-a30b-e01d1d633ad7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:37.901311 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e403c2-93db-47d1-a30b-e01d1d633ad7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:37.913073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 231/452] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 02:59:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:37.913937 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e403c2-93db-47d1-a30b-e01d1d633ad7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:37.981101 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a12e7-ebac-4f3f-a358-f861ed355851 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:38.011741 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e403c2-93db-47d1-a30b-e01d1d633ad7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:38.013783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195e861-5928-4b96-9ab4-b2feed98f0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:38.014382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195e861-5928-4b96-9ab4-b2feed98f0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:38.014662 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195e861-5928-4b96-9ab4-b2feed98f0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:38.014968 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195e861-5928-4b96-9ab4-b2feed98f0c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:38.015613 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195e861-5928-4b96-9ab4-b2feed98f0c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:38.015920 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195e861-5928-4b96-9ab4-b2feed98f0c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:38.017293 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195e861-5928-4b96-9ab4-b2feed98f0c4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:38.028774 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 222/453] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:38.029559 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195e861-5928-4b96-9ab4-b2feed98f0c4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:38.045123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b921f96-fe8a-4596-9e6b-c918dd31bc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:38.045731 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b921f96-fe8a-4596-9e6b-c918dd31bc49 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:38.046014 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b921f96-fe8a-4596-9e6b-c918dd31bc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:38.046285 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b921f96-fe8a-4596-9e6b-c918dd31bc49 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:38.047041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b921f96-fe8a-4596-9e6b-c918dd31bc49 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:38.047347 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b921f96-fe8a-4596-9e6b-c918dd31bc49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:38.048763 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b921f96-fe8a-4596-9e6b-c918dd31bc49 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:38.065675 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 232/454] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Oct 17 02:59:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:38.066464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b921f96-fe8a-4596-9e6b-c918dd31bc49 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:38.093099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:38.094443 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:38.129532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195e861-5928-4b96-9ab4-b2feed98f0c4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:38.162657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b1265-dc45-4486-a9f8-1ef0940bbe8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:38.163239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b1265-dc45-4486-a9f8-1ef0940bbe8a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:38.163517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b1265-dc45-4486-a9f8-1ef0940bbe8a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:38.163789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b1265-dc45-4486-a9f8-1ef0940bbe8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:38.170149 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b921f96-fe8a-4596-9e6b-c918dd31bc49 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:38.173481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b1265-dc45-4486-a9f8-1ef0940bbe8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:38.178776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-b17b1265-dc45-4486-a9f8-1ef0940bbe8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:38.178776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:38.178776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:38.178776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:38.178776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:38.178776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:38.178776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:38.178776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:38.178776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:38.178776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:38.178776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:38.178776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:38.178776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:38.178776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:38.178776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:38.178776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:38.179868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:38.179868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:38.179868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:38.179868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:38.179868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:38.179868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:38.179868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:38.179868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:38.179868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:38.179868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:38.179868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:38.179868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:38.179868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:38.179868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:38.179868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:38.179868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:38.180823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:38.180823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:38.180823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:38.180823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:38.180823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:38.180823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:38.180823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:38.180823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:38.180823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:38.180823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:38.180823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:38.180823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:38.180823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 223/455] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Oct 17 02:59:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:38.180823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b1265-dc45-4486-a9f8-1ef0940bbe8a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:38.205161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7e968-464d-45c4-9732-effd69732165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:38.205849 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7e968-464d-45c4-9732-effd69732165 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:38.206133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7e968-464d-45c4-9732-effd69732165 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:38.206400 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7e968-464d-45c4-9732-effd69732165 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:38.207081 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7e968-464d-45c4-9732-effd69732165 None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:38.207554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7e968-464d-45c4-9732-effd69732165 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:38.209041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7e968-464d-45c4-9732-effd69732165 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:38.216906 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 233/456] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Oct 17 02:59:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:38.217711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7e968-464d-45c4-9732-effd69732165 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:38.284306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b1265-dc45-4486-a9f8-1ef0940bbe8a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:38.305906 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:38.310616 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:38.319799 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e7e5b-5bd4-44dc-bc15-732d5a1148d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:38.320481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e7e5b-5bd4-44dc-bc15-732d5a1148d2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:38.320827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e7e5b-5bd4-44dc-bc15-732d5a1148d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:38.321177 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e7e5b-5bd4-44dc-bc15-732d5a1148d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:38.321967 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e7e5b-5bd4-44dc-bc15-732d5a1148d2 None admin] RBAC: Adding query filter params (role.id=78dc08989b32455389b21096f5d57e17, scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2, user.id=01637fc07ba846aebcb42325bed46216) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:38.322355 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e7e5b-5bd4-44dc-bc15-732d5a1148d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:38.323046 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7e968-464d-45c4-9732-effd69732165 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:38.323974 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e7e5b-5bd4-44dc-bc15-732d5a1148d2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:38.329991 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 224/457] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 02:59:38 2022] GET /identity/v3/role_assignments?user.id=01637fc07ba846aebcb42325bed46216&scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2&role.id=78dc08989b32455389b21096f5d57e17 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:38.330694 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e7e5b-5bd4-44dc-bc15-732d5a1148d2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:38.361540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd0539-9ead-4baf-8089-b1bf92a0db8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:38.362299 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd0539-9ead-4baf-8089-b1bf92a0db8b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:38.362744 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd0539-9ead-4baf-8089-b1bf92a0db8b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:38.363033 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd0539-9ead-4baf-8089-b1bf92a0db8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:38.370880 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd0539-9ead-4baf-8089-b1bf92a0db8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:38.372471 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd0539-9ead-4baf-8089-b1bf92a0db8b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:38.385134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-b5cd0539-9ead-4baf-8089-b1bf92a0db8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:38.385134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:38.385134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:38.385134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:38.385134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:38.385134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:38.385134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:38.385134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:38.385134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:38.385134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:38.385134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:38.385134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:38.385134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 02:59:38.385134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 02:59:38.385134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 02:59:38.386251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 02:59:38.386251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:38.386251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:38.386251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:38.386251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:38.386251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:38.386251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:38.386251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:38.386251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:38.386251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:38.386251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:38.386251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:38.386251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:38.386251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:38.386251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:38.386251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:38.386251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:38.387491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 02:59:38.387491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 02:59:38.387491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 02:59:38.387491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 02:59:38.387491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 02:59:38.387491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 02:59:38.387491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:38.387491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:38.393707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 234/458] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:38.393707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd0539-9ead-4baf-8089-b1bf92a0db8b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:38.460661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e7e5b-5bd4-44dc-bc15-732d5a1148d2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:38.496848 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa7576-40e7-4753-b553-5d85e685f55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:38.497451 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa7576-40e7-4753-b553-5d85e685f55f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:38.497732 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa7576-40e7-4753-b553-5d85e685f55f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:38.498002 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa7576-40e7-4753-b553-5d85e685f55f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:38.510747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd0539-9ead-4baf-8089-b1bf92a0db8b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:38.549231 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e36b57-0078-4b22-8971-55d2f438f247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:38.549856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e36b57-0078-4b22-8971-55d2f438f247 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:38.550141 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e36b57-0078-4b22-8971-55d2f438f247 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:38.550415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e36b57-0078-4b22-8971-55d2f438f247 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:38.551143 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e36b57-0078-4b22-8971-55d2f438f247 None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:38.551448 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e36b57-0078-4b22-8971-55d2f438f247 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:38.552829 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e36b57-0078-4b22-8971-55d2f438f247 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:38.561412 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 235/459] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Oct 17 02:59:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:38.562043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e36b57-0078-4b22-8971-55d2f438f247 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:38.569048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:38.572789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:38.584163 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa7576-40e7-4753-b553-5d85e685f55f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:38.585817 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa7576-40e7-4753-b553-5d85e685f55f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:38.669570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e36b57-0078-4b22-8971-55d2f438f247 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:38.701330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 225/460] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:59:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 02:59:38.705859 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa7576-40e7-4753-b553-5d85e685f55f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:38.712941 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8fc25-9018-47b5-9799-37354857da54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:38.713516 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8fc25-9018-47b5-9799-37354857da54 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:38.713791 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8fc25-9018-47b5-9799-37354857da54 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:38.714060 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8fc25-9018-47b5-9799-37354857da54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:38.725268 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8fc25-9018-47b5-9799-37354857da54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:38.726709 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8fc25-9018-47b5-9799-37354857da54 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:38.741099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-1de8fc25-9018-47b5-9799-37354857da54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:38.741099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:38.741099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:38.741099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:38.741099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:38.741099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:38.741099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:38.741099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:38.741099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:38.741099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:38.741099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:38.741099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:38.741099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:38.741099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:38.741099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:38.741099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:38.742211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:38.742211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:38.742211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:38.742211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:38.742211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:38.742211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:38.742211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:38.742211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:38.742211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:38.742211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:38.742211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:38.742211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:38.742211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:38.742211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:38.742211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:38.742211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:38.742211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:38.743106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:38.743106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:38.743106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:38.743106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 236/461] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:38.743339 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8fc25-9018-47b5-9799-37354857da54 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:38.803048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:38.807649 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:38.815849 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa7576-40e7-4753-b553-5d85e685f55f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:38.852035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f7853-3092-464c-9ecb-4d50e9e95ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:38.852637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f7853-3092-464c-9ecb-4d50e9e95ecb None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:38.852915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f7853-3092-464c-9ecb-4d50e9e95ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:38.853183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f7853-3092-464c-9ecb-4d50e9e95ecb None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:38.853525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8fc25-9018-47b5-9799-37354857da54 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:38.853867 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f7853-3092-464c-9ecb-4d50e9e95ecb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:38.854168 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f7853-3092-464c-9ecb-4d50e9e95ecb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:38.855632 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f7853-3092-464c-9ecb-4d50e9e95ecb None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:38.872668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 226/462] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 02:59:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:38.872668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f7853-3092-464c-9ecb-4d50e9e95ecb None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:38.895762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3bace1-d6d2-441e-a188-043e676dbcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:38.896400 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3bace1-d6d2-441e-a188-043e676dbcb1 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:38.896681 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3bace1-d6d2-441e-a188-043e676dbcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:38.896954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3bace1-d6d2-441e-a188-043e676dbcb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:38.907664 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3bace1-d6d2-441e-a188-043e676dbcb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:38.909311 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3bace1-d6d2-441e-a188-043e676dbcb1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:38.923627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-fe3bace1-d6d2-441e-a188-043e676dbcb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:38.923627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:38.923627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:38.923627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:38.923627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:38.923627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:38.923627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:38.923627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:38.923627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:38.923627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:38.923627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:38.923627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:38.923627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:38.923627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:38.923627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:38.923627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:38.929736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:38.929736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:38.929736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:38.929736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:38.929736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:38.929736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:38.929736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:38.929736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:38.929736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:38.929736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:38.929736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:38.929736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:38.929736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:38.929736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:38.929736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:38.929736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:38.929736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:38.930834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:38.930834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:38.930834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:38.930834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 237/463] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:38.930834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3bace1-d6d2-441e-a188-043e676dbcb1 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:38.984305 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f7853-3092-464c-9ecb-4d50e9e95ecb None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:39.018958 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99975b6-ed45-457d-960f-76ad6375ed24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:39.019599 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99975b6-ed45-457d-960f-76ad6375ed24 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:39.020426 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99975b6-ed45-457d-960f-76ad6375ed24 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:39.020706 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99975b6-ed45-457d-960f-76ad6375ed24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:39.032277 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99975b6-ed45-457d-960f-76ad6375ed24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:39.033767 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99975b6-ed45-457d-960f-76ad6375ed24 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:39.045937 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3bace1-d6d2-441e-a188-043e676dbcb1 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:39.048940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-e99975b6-ed45-457d-960f-76ad6375ed24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:39.048940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:39.048940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:39.048940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:39.048940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:39.048940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:39.048940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:39.048940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:39.048940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:39.048940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:39.048940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:39.048940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:39.048940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:39.048940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:39.048940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:39.048940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:39.050005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:39.050005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:39.050005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:39.050005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:39.050005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:39.050005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:39.050005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:39.050005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:39.050005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:39.050005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:39.050005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:39.050005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:39.050005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:39.050005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:39.050005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:39.050005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:39.050005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:39.050851 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:39.050851 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:39.050851 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:39.050851 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 227/464] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:39.050851 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99975b6-ed45-457d-960f-76ad6375ed24 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:39.056992 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:39.060121 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:39.086300 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e8980-180c-4cba-a741-24e73b085c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:39.086942 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e8980-180c-4cba-a741-24e73b085c9c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:39.087225 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e8980-180c-4cba-a741-24e73b085c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:39.087497 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991e8980-180c-4cba-a741-24e73b085c9c None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:39.088138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e8980-180c-4cba-a741-24e73b085c9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:39.088441 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e8980-180c-4cba-a741-24e73b085c9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:39.089888 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991e8980-180c-4cba-a741-24e73b085c9c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:39.106242 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 238/465] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:39.106242 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e8980-180c-4cba-a741-24e73b085c9c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:39.169648 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99975b6-ed45-457d-960f-76ad6375ed24 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:39.205019 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528038eb-d33a-4b30-9f14-5f92e1cac1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:39.205916 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528038eb-d33a-4b30-9f14-5f92e1cac1b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:39.206199 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528038eb-d33a-4b30-9f14-5f92e1cac1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:39.206475 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528038eb-d33a-4b30-9f14-5f92e1cac1b8 None admin] PATH_INFO: `/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/eceda8b3a8ba4233851d0138d8f4d06a/roles/78dc08989b32455389b21096f5d57e17` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:39.208990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991e8980-180c-4cba-a741-24e73b085c9c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:39.222704 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528038eb-d33a-4b30-9f14-5f92e1cac1b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=89b0ee95955f465eb9ee5b8ae337d9c2, user_id=eceda8b3a8ba4233851d0138d8f4d06a, role_id=78dc08989b32455389b21096f5d57e17)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:39.224507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528038eb-d33a-4b30-9f14-5f92e1cac1b8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:39.239706 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 228/466] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 02:59:39 2022] PUT /identity/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/eceda8b3a8ba4233851d0138d8f4d06a/roles/78dc08989b32455389b21096f5d57e17 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:59:39.240484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528038eb-d33a-4b30-9f14-5f92e1cac1b8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:39.262356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270af84-61a4-49d5-89b1-d6bf3bf066db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:39.263220 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270af84-61a4-49d5-89b1-d6bf3bf066db None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:39.263407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270af84-61a4-49d5-89b1-d6bf3bf066db None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:39.263820 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270af84-61a4-49d5-89b1-d6bf3bf066db None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:39.264580 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270af84-61a4-49d5-89b1-d6bf3bf066db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:39.264890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270af84-61a4-49d5-89b1-d6bf3bf066db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:39.266367 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270af84-61a4-49d5-89b1-d6bf3bf066db None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:39.284314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 239/467] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:39.285942 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270af84-61a4-49d5-89b1-d6bf3bf066db None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:39.323383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:39.325820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:39.398436 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528038eb-d33a-4b30-9f14-5f92e1cac1b8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:39.420831 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270af84-61a4-49d5-89b1-d6bf3bf066db None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:39.438509 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25523310-5acc-4887-a449-f8caf886101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:39.439170 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25523310-5acc-4887-a449-f8caf886101c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:39.439455 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25523310-5acc-4887-a449-f8caf886101c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:39.439735 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25523310-5acc-4887-a449-f8caf886101c None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:39.440385 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25523310-5acc-4887-a449-f8caf886101c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:39.440691 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25523310-5acc-4887-a449-f8caf886101c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:39.442619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25523310-5acc-4887-a449-f8caf886101c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:39.455129 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 229/468] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:39.456266 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25523310-5acc-4887-a449-f8caf886101c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:39.471808 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14f830-9a44-4515-be82-44b7c651e04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:39.473528 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14f830-9a44-4515-be82-44b7c651e04b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:39.474320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14f830-9a44-4515-be82-44b7c651e04b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:39.474985 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14f830-9a44-4515-be82-44b7c651e04b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:39.483584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14f830-9a44-4515-be82-44b7c651e04b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:39.485389 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14f830-9a44-4515-be82-44b7c651e04b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:39.499534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-6f14f830-9a44-4515-be82-44b7c651e04b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 17 02:59:39.499534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:39.499534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:39.499534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:39.499534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:39.499534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:39.499534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:39.499534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:39.499534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:39.499534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:39.499534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:39.499534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:39.499534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 02:59:39.499534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 02:59:39.499534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 02:59:39.499534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:39.500447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:39.501343 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:39.501343 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:39.501343 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 02:59:39.501343 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 02:59:39.501343 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 02:59:39.501343 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 02:59:39.501343 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 02:59:39.501343 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 02:59:39.501343 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 17 02:59:39.501343 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:39.504958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 240/469] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:39.506813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14f830-9a44-4515-be82-44b7c651e04b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:39.569807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25523310-5acc-4887-a449-f8caf886101c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:39.586534 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:39.618502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35035a74-b6bf-477b-8d65-cb877c1702bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:39.619168 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35035a74-b6bf-477b-8d65-cb877c1702bd None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:39.619445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35035a74-b6bf-477b-8d65-cb877c1702bd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:39.619716 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35035a74-b6bf-477b-8d65-cb877c1702bd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:39.627675 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:39.631484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35035a74-b6bf-477b-8d65-cb877c1702bd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:39.633330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35035a74-b6bf-477b-8d65-cb877c1702bd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:39.639739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14f830-9a44-4515-be82-44b7c651e04b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:39.648192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-35035a74-b6bf-477b-8d65-cb877c1702bd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 02:59:39.648192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:39.648192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:39.648192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:39.648192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:39.648192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:39.648192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:39.648192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:39.648192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:39.648192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:39.648192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:39.648192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:39.648192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 02:59:39.648192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 02:59:39.648192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 02:59:39.648192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:39.649092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:39.650007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:39.650007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:39.650007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 02:59:39.650007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 02:59:39.650007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 02:59:39.650007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 02:59:39.650007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 02:59:39.650007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 02:59:39.650007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 02:59:39.650007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:39.656800 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 230/470] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Oct 17 02:59:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:39.657981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35035a74-b6bf-477b-8d65-cb877c1702bd None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:39.679692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6b359-1902-4501-b40e-25e62ec3a088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:39.680330 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6b359-1902-4501-b40e-25e62ec3a088 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:39.680626 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6b359-1902-4501-b40e-25e62ec3a088 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:39.680881 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6b359-1902-4501-b40e-25e62ec3a088 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:39.689885 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6b359-1902-4501-b40e-25e62ec3a088 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:39.691868 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6b359-1902-4501-b40e-25e62ec3a088 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:39.705978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-46e6b359-1902-4501-b40e-25e62ec3a088 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:39.705978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:39.705978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:39.705978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:39.705978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:39.705978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:39.705978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:39.705978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:39.705978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:39.705978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:39.705978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:39.705978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:39.705978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 02:59:39.705978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 02:59:39.705978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 02:59:39.706890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 02:59:39.706890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:39.706890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:39.706890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:39.706890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:39.706890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:39.706890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:39.706890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:39.706890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:39.706890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:39.706890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:39.706890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:39.706890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:39.706890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:39.706890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:39.706890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:39.706890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:39.708289 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 02:59:39.708289 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 02:59:39.708289 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 02:59:39.708289 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 02:59:39.708289 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 02:59:39.708289 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 02:59:39.708289 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:39.708289 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:39.708289 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 241/471] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:39.709232 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6b359-1902-4501-b40e-25e62ec3a088 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:39.773815 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35035a74-b6bf-477b-8d65-cb877c1702bd None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:39.811189 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ccdc5-cf98-4bc4-8f16-2eee13d72eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:39.811909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ccdc5-cf98-4bc4-8f16-2eee13d72eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:39.812269 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ccdc5-cf98-4bc4-8f16-2eee13d72eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:39.812755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ccdc5-cf98-4bc4-8f16-2eee13d72eb0 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:39.813573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ccdc5-cf98-4bc4-8f16-2eee13d72eb0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:39.813956 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ccdc5-cf98-4bc4-8f16-2eee13d72eb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:39.815531 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ccdc5-cf98-4bc4-8f16-2eee13d72eb0 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:39.828244 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6b359-1902-4501-b40e-25e62ec3a088 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:39.839073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 231/472] 10.222.0.27 () {56 vars in 1137 bytes} [Mon Oct 17 02:59:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:39.840058 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ccdc5-cf98-4bc4-8f16-2eee13d72eb0 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:39.870386 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df282744-40b3-4130-b9e9-76666ad9c188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:39.871174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df282744-40b3-4130-b9e9-76666ad9c188 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:39.871454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df282744-40b3-4130-b9e9-76666ad9c188 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:39.871648 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:39.871752 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df282744-40b3-4130-b9e9-76666ad9c188 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:39.872478 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df282744-40b3-4130-b9e9-76666ad9c188 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:39.872855 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df282744-40b3-4130-b9e9-76666ad9c188 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:39.874170 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df282744-40b3-4130-b9e9-76666ad9c188 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:39.899144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 242/473] 10.222.0.27 () {56 vars in 1139 bytes} [Mon Oct 17 02:59:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:39.899144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df282744-40b3-4130-b9e9-76666ad9c188 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:39.908317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:39.964970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ccdc5-cf98-4bc4-8f16-2eee13d72eb0 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:40.002759 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba586b5-eb16-4257-b0d1-d00b9d782164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:40.003739 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba586b5-eb16-4257-b0d1-d00b9d782164 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:40.004365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba586b5-eb16-4257-b0d1-d00b9d782164 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:40.004834 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba586b5-eb16-4257-b0d1-d00b9d782164 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:40.006070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba586b5-eb16-4257-b0d1-d00b9d782164 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:40.006595 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba586b5-eb16-4257-b0d1-d00b9d782164 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:40.008523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba586b5-eb16-4257-b0d1-d00b9d782164 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:40.017860 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df282744-40b3-4130-b9e9-76666ad9c188 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:40.022801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 232/474] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 02:59:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:40.024043 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba586b5-eb16-4257-b0d1-d00b9d782164 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:40.060352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba75e6-764d-433a-a121-3d0cfe4dfe7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:40.060962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba75e6-764d-433a-a121-3d0cfe4dfe7b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:40.061241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba75e6-764d-433a-a121-3d0cfe4dfe7b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:40.062282 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba75e6-764d-433a-a121-3d0cfe4dfe7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:40.122805 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:40.137599 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba586b5-eb16-4257-b0d1-d00b9d782164 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:40.147007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba75e6-764d-433a-a121-3d0cfe4dfe7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:40.148834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba75e6-764d-433a-a121-3d0cfe4dfe7b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:40.162491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:40.177525 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d882ddb-9762-4a13-a517-2b70b2e69706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:40.178982 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d882ddb-9762-4a13-a517-2b70b2e69706 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:40.179563 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d882ddb-9762-4a13-a517-2b70b2e69706 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:40.180039 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d882ddb-9762-4a13-a517-2b70b2e69706 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:40.268100 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d882ddb-9762-4a13-a517-2b70b2e69706 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:40.270084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d882ddb-9762-4a13-a517-2b70b2e69706 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:40.288109 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 243/475] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:59:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 02:59:40.289067 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba75e6-764d-433a-a121-3d0cfe4dfe7b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:40.371557 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:40.390011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 233/476] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:59:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 02:59:40.391270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d882ddb-9762-4a13-a517-2b70b2e69706 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:40.412838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba75e6-764d-433a-a121-3d0cfe4dfe7b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:40.427044 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:40.461502 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d94ba-631b-44af-9b37-24606434940a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:40.462393 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d94ba-631b-44af-9b37-24606434940a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:40.462678 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d94ba-631b-44af-9b37-24606434940a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:40.462977 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d94ba-631b-44af-9b37-24606434940a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:40.463716 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d94ba-631b-44af-9b37-24606434940a None admin] RBAC: Adding query filter params (role.id=78dc08989b32455389b21096f5d57e17, scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2, user.id=67abf21b99d349dfa7d88a0e18be1fed) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:40.464024 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d94ba-631b-44af-9b37-24606434940a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:40.466129 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4d94ba-631b-44af-9b37-24606434940a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:40.474344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 244/477] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 02:59:40 2022] GET /identity/v3/role_assignments?user.id=67abf21b99d349dfa7d88a0e18be1fed&scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2&role.id=78dc08989b32455389b21096f5d57e17 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:40.475472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d94ba-631b-44af-9b37-24606434940a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:40.515676 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d882ddb-9762-4a13-a517-2b70b2e69706 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:40.578921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b079fdd-5242-435f-b61f-f38f25412be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:40.579639 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b079fdd-5242-435f-b61f-f38f25412be8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:40.579925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b079fdd-5242-435f-b61f-f38f25412be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:40.580895 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b079fdd-5242-435f-b61f-f38f25412be8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:40.596718 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b079fdd-5242-435f-b61f-f38f25412be8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:40.598601 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b079fdd-5242-435f-b61f-f38f25412be8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:40.620859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d94ba-631b-44af-9b37-24606434940a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:40.621073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2b079fdd-5242-435f-b61f-f38f25412be8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:40.621073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:40.621073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:40.621073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:40.621073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:40.621073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:40.621073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:40.621073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:40.621073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:40.621073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:40.621073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:40.621073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:40.621073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:40.621073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:40.621073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:40.621073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:40.622139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:40.622139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:40.622139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:40.622139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:40.622139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:40.622139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:40.622139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:40.622139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:40.622139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:40.622139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:40.622139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:40.622139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:40.622139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:40.622139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:40.622139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:40.622139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:40.622139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:40.623225 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:40.623225 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:40.623225 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:40.623225 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 234/478] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:40.627880 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324467e-93f2-4611-ae15-c5cbdbdca89f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:40.627880 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324467e-93f2-4611-ae15-c5cbdbdca89f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:40.627880 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324467e-93f2-4611-ae15-c5cbdbdca89f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:40.627880 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 235/479] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:40.632774 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324467e-93f2-4611-ae15-c5cbdbdca89f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:40.669743 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272db55-3e4d-4928-9cbc-6fe47c4419f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:40.670425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272db55-3e4d-4928-9cbc-6fe47c4419f8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:40.670721 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272db55-3e4d-4928-9cbc-6fe47c4419f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:40.671074 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272db55-3e4d-4928-9cbc-6fe47c4419f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:40.683732 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272db55-3e4d-4928-9cbc-6fe47c4419f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:40.685280 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272db55-3e4d-4928-9cbc-6fe47c4419f8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:40.694898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:40.704415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e272db55-3e4d-4928-9cbc-6fe47c4419f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:40.704415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:40.704415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:40.704415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:40.704415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:40.704415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:40.704415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:40.704415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:40.704415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:40.704415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:40.704415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:40.704415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:40.704415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:40.704415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:40.704415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:40.704415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:40.705492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:40.705492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:40.705492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:40.705492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:40.705492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:40.705492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:40.705492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:40.705492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:40.705492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:40.705492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:40.705492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:40.705492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:40.705492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:40.705492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:40.705492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:40.705492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:40.705492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:40.706680 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:40.706680 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:40.706680 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:40.708947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 245/480] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:40.709967 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5fd15-ea35-4f95-b435-95148acc8be1 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:40.710250 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5fd15-ea35-4f95-b435-95148acc8be1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:40.710543 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5fd15-ea35-4f95-b435-95148acc8be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:40.730746 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1bd5fd15-ea35-4f95-b435-95148acc8be1 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:40.731981 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1bd5fd15-ea35-4f95-b435-95148acc8be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:40.754178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:40.768636 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324467e-93f2-4611-ae15-c5cbdbdca89f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:40.811382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee1409-8c0d-4c4a-9da7-5ecd1a8a24dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:40.811985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee1409-8c0d-4c4a-9da7-5ecd1a8a24dd None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:40.812261 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee1409-8c0d-4c4a-9da7-5ecd1a8a24dd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:40.812591 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee1409-8c0d-4c4a-9da7-5ecd1a8a24dd None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:40.813225 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee1409-8c0d-4c4a-9da7-5ecd1a8a24dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:40.813594 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee1409-8c0d-4c4a-9da7-5ecd1a8a24dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:40.815120 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee1409-8c0d-4c4a-9da7-5ecd1a8a24dd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:40.816433 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 246/481] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:40.823886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5fd15-ea35-4f95-b435-95148acc8be1 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:40.829362 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 236/482] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:40.842890 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee1409-8c0d-4c4a-9da7-5ecd1a8a24dd None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:41.008576 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5fd15-ea35-4f95-b435-95148acc8be1 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:41.016461 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee1409-8c0d-4c4a-9da7-5ecd1a8a24dd None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:41.019692 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:41.054332 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe6f1b-400f-490a-9fb2-304a7918020c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:41.055007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe6f1b-400f-490a-9fb2-304a7918020c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:41.055350 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe6f1b-400f-490a-9fb2-304a7918020c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:41.055630 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe6f1b-400f-490a-9fb2-304a7918020c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:41.068475 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe6f1b-400f-490a-9fb2-304a7918020c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:41.069937 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe6f1b-400f-490a-9fb2-304a7918020c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:41.084410 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec1301-17a9-4c37-b8f1-4b7cb2f712e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:41.084951 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec1301-17a9-4c37-b8f1-4b7cb2f712e0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:41.085231 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec1301-17a9-4c37-b8f1-4b7cb2f712e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:41.085566 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec1301-17a9-4c37-b8f1-4b7cb2f712e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:41.086221 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec1301-17a9-4c37-b8f1-4b7cb2f712e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:41.086525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec1301-17a9-4c37-b8f1-4b7cb2f712e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:41.087923 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec1301-17a9-4c37-b8f1-4b7cb2f712e0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:41.090945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-89fe6f1b-400f-490a-9fb2-304a7918020c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:41.090945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:41.090945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:41.090945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:41.090945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:41.090945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:41.090945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:41.090945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:41.090945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:41.090945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:41.090945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:41.090945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:41.090945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 02:59:41.090945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 02:59:41.090945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 02:59:41.091864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 02:59:41.091864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:41.091864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:41.091864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:41.091864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:41.091864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:41.091864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:41.091864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:41.091864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:41.091864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:41.091864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:41.091864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:41.091864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:41.091864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:41.091864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:41.091864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:41.091864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:41.092724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 02:59:41.092724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 02:59:41.092724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 02:59:41.092724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 02:59:41.092724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 02:59:41.092724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 02:59:41.092724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:41.092724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:41.092724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 237/483] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:41.100961 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe6f1b-400f-490a-9fb2-304a7918020c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:41.105882 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 247/484] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:41.111009 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec1301-17a9-4c37-b8f1-4b7cb2f712e0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:41.140259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:41.233780 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec1301-17a9-4c37-b8f1-4b7cb2f712e0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:41.250495 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe6f1b-400f-490a-9fb2-304a7918020c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:41.272360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc31381-ca78-4108-afc7-ed8084f78641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:41.273379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc31381-ca78-4108-afc7-ed8084f78641 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:41.273905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc31381-ca78-4108-afc7-ed8084f78641 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:41.274355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc31381-ca78-4108-afc7-ed8084f78641 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:41.281788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc31381-ca78-4108-afc7-ed8084f78641 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:41.283279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc31381-ca78-4108-afc7-ed8084f78641 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:41.298229 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e22973-3597-465b-87dd-c522fac0fbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:41.298845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ddc31381-ca78-4108-afc7-ed8084f78641 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:41.298845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:41.298845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:41.298845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:41.298845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:41.298845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:41.298845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:41.298845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:41.298845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:41.298845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:41.298845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:41.298845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:41.298845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 02:59:41.298845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 02:59:41.298845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 02:59:41.299726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 02:59:41.299726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:41.299726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:41.299726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:41.299726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:41.299726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:41.299726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:41.299726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:41.299726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:41.299726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:41.299726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:41.299726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:41.299726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:41.299726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:41.299726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:41.299726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:41.299726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:41.300652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 02:59:41.300652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 02:59:41.300652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 02:59:41.300652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 02:59:41.300652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 02:59:41.300652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 02:59:41.300652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:41.300652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:41.300652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 248/485] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:41.301601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc31381-ca78-4108-afc7-ed8084f78641 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:41.302249 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e22973-3597-465b-87dd-c522fac0fbc2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:41.302766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e22973-3597-465b-87dd-c522fac0fbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:41.303257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e22973-3597-465b-87dd-c522fac0fbc2 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:41.304361 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e22973-3597-465b-87dd-c522fac0fbc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:41.305775 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e22973-3597-465b-87dd-c522fac0fbc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:41.307455 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e22973-3597-465b-87dd-c522fac0fbc2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:41.320380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 238/486] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 02:59:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:41.321261 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e22973-3597-465b-87dd-c522fac0fbc2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:41.346748 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:41.395608 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:41.417529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc31381-ca78-4108-afc7-ed8084f78641 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:41.426774 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e22973-3597-465b-87dd-c522fac0fbc2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:41.455147 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279cb7b-4701-4624-905b-e19e45c1d3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:41.456263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279cb7b-4701-4624-905b-e19e45c1d3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:41.456855 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279cb7b-4701-4624-905b-e19e45c1d3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:41.457863 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279cb7b-4701-4624-905b-e19e45c1d3d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:41.462097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06865915-81c6-40d4-873b-7d4b3646c0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:41.462713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06865915-81c6-40d4-873b-7d4b3646c0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:41.463059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06865915-81c6-40d4-873b-7d4b3646c0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:41.463338 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06865915-81c6-40d4-873b-7d4b3646c0a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:41.464054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06865915-81c6-40d4-873b-7d4b3646c0a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:41.464337 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06865915-81c6-40d4-873b-7d4b3646c0a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:41.465975 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06865915-81c6-40d4-873b-7d4b3646c0a4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:41.470970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279cb7b-4701-4624-905b-e19e45c1d3d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:41.477661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-8279cb7b-4701-4624-905b-e19e45c1d3d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:59:41.477661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:41.477661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:41.477661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:41.477661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:41.477661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:41.477661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:41.477661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:41.477661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:41.477661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:41.477661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:41.477661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:41.477661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:41.477661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:41.477661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:41.477661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:41.478556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:41.478556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:41.478556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:41.478556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:41.478556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:41.478556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:41.478556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:41.478556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:41.478556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:41.478556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:41.478556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:41.478556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:41.478556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:41.478556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:41.478556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:41.478556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:41.479383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:41.479383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:41.479383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:41.479383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:41.479383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:41.479383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:41.479383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:41.479383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:41.479383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:41.479383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:41.479383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:59:41.479383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:41.479995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 239/487] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 02:59:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:41.480065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06865915-81c6-40d4-873b-7d4b3646c0a4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:41.482016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 249/488] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 02:59:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:41.490946 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279cb7b-4701-4624-905b-e19e45c1d3d5 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:41.591177 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:41.615422 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279cb7b-4701-4624-905b-e19e45c1d3d5 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:41.616406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06865915-81c6-40d4-873b-7d4b3646c0a4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:41.655207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6531c1-536c-40da-98bf-8b81d40936c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:41.655823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6531c1-536c-40da-98bf-8b81d40936c8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:41.656106 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6531c1-536c-40da-98bf-8b81d40936c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:41.656384 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6531c1-536c-40da-98bf-8b81d40936c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:41.657113 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6531c1-536c-40da-98bf-8b81d40936c8 None admin] RBAC: Adding query filter params (role.id=78dc08989b32455389b21096f5d57e17, scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2, user.id=a8aa29639a3143d2a270e3248d2205d1) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:41.657446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6531c1-536c-40da-98bf-8b81d40936c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:41.658894 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6531c1-536c-40da-98bf-8b81d40936c8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:41.661629 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043fd9d4-ad7f-48de-b6bb-232ca3008870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:41.662954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fd9d4-ad7f-48de-b6bb-232ca3008870 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:41.663688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fd9d4-ad7f-48de-b6bb-232ca3008870 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:41.664123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fd9d4-ad7f-48de-b6bb-232ca3008870 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:41.665005 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043fd9d4-ad7f-48de-b6bb-232ca3008870 None admin] RBAC: Adding query filter params (role.id=78dc08989b32455389b21096f5d57e17, scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2, user.id=526933e6ad004493b071f5da5caa09a3) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:41.665364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 240/489] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 02:59:41 2022] GET /identity/v3/role_assignments?user.id=a8aa29639a3143d2a270e3248d2205d1&scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2&role.id=78dc08989b32455389b21096f5d57e17 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:41.665660 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043fd9d4-ad7f-48de-b6bb-232ca3008870 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:41.666136 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6531c1-536c-40da-98bf-8b81d40936c8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:41.667609 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043fd9d4-ad7f-48de-b6bb-232ca3008870 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:41.669811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:41.682771 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 250/490] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 02:59:41 2022] GET /identity/v3/role_assignments?user.id=526933e6ad004493b071f5da5caa09a3&scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2&role.id=78dc08989b32455389b21096f5d57e17 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:41.684560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a164d0-84de-4721-83d9-82948f5cd21d None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:41.685045 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a164d0-84de-4721-83d9-82948f5cd21d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:41.685470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a164d0-84de-4721-83d9-82948f5cd21d None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:41.687131 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 251/491] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:41.694244 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1223c1c-4e24-4972-980e-55829401c46c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:41.694769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1223c1c-4e24-4972-980e-55829401c46c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:41.695229 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1223c1c-4e24-4972-980e-55829401c46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:41.727635 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e1223c1c-4e24-4972-980e-55829401c46c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:41.729346 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e1223c1c-4e24-4972-980e-55829401c46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:41.862498 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6531c1-536c-40da-98bf-8b81d40936c8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:41.889733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 252/492] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:41.916993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44fe19-fb01-4768-9db6-e776085de83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:41.917607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44fe19-fb01-4768-9db6-e776085de83d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:41.917883 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44fe19-fb01-4768-9db6-e776085de83d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:41.918150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44fe19-fb01-4768-9db6-e776085de83d None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:41.918813 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44fe19-fb01-4768-9db6-e776085de83d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:41.919167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44fe19-fb01-4768-9db6-e776085de83d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:41.920546 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44fe19-fb01-4768-9db6-e776085de83d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:41.931907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 241/493] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:41.937056 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1223c1c-4e24-4972-980e-55829401c46c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:42.053597 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1223c1c-4e24-4972-980e-55829401c46c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:42.083903 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:42.093376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268b6de-366e-4598-abb3-2d0782daeb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:42.094005 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268b6de-366e-4598-abb3-2d0782daeb6e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:42.094285 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268b6de-366e-4598-abb3-2d0782daeb6e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:42.094554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268b6de-366e-4598-abb3-2d0782daeb6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:42.105466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268b6de-366e-4598-abb3-2d0782daeb6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:42.106953 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268b6de-366e-4598-abb3-2d0782daeb6e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:42.122904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-1268b6de-366e-4598-abb3-2d0782daeb6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:42.122904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:42.122904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:42.122904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:42.122904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:42.122904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:42.122904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:42.122904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:42.122904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:42.122904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:42.122904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:42.122904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:42.122904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:42.122904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:42.122904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:42.122904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:42.124270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:42.124270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:42.124270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:42.124270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:42.124270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:42.124270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:42.124270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:42.124270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:42.124270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:42.124270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:42.124270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:42.124270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:42.124270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:42.124270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:42.124270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:42.124270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:42.124270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:42.125380 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:42.125380 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:42.125380 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:42.128506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 253/494] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:42.131958 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44fe19-fb01-4768-9db6-e776085de83d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:42.153902 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:42.223534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268b6de-366e-4598-abb3-2d0782daeb6e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:42.282985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44fe19-fb01-4768-9db6-e776085de83d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:42.329239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49bbff-e0e0-4c16-a5ec-ff3805288df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:42.330251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49bbff-e0e0-4c16-a5ec-ff3805288df4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:42.330747 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49bbff-e0e0-4c16-a5ec-ff3805288df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:42.331260 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49bbff-e0e0-4c16-a5ec-ff3805288df4 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:42.332387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49bbff-e0e0-4c16-a5ec-ff3805288df4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:42.332890 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49bbff-e0e0-4c16-a5ec-ff3805288df4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:42.334574 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49bbff-e0e0-4c16-a5ec-ff3805288df4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:42.353036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 242/495] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:42.364318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49bbff-e0e0-4c16-a5ec-ff3805288df4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:42.405828 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268b6de-366e-4598-abb3-2d0782daeb6e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:42.472499 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52285bfb-3b9e-46a8-b8d3-56171f237a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:42.473665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52285bfb-3b9e-46a8-b8d3-56171f237a45 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:42.473944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52285bfb-3b9e-46a8-b8d3-56171f237a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:42.474215 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52285bfb-3b9e-46a8-b8d3-56171f237a45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:42.482794 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:42.496718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52285bfb-3b9e-46a8-b8d3-56171f237a45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:42.507223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-52285bfb-3b9e-46a8-b8d3-56171f237a45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:42.507223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:42.507223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:42.507223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:42.507223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:42.507223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:42.507223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:42.507223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:42.507223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:42.507223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:42.507223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:42.507223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:42.507223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:42.507223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:42.507223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:42.507223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:42.508157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:42.508157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:42.508157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:42.508157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:42.508157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:42.508157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:42.508157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:42.508157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:42.508157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:42.508157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:42.508157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:42.508157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:42.508157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:42.508157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:42.508157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:42.508157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:42.509061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:42.509061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:42.509061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:42.509061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:42.509061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:42.509061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:42.509061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:42.509061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:42.509061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:42.509061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:42.509061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:42.509061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:42.509061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 254/496] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Oct 17 02:59:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:42.516144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52285bfb-3b9e-46a8-b8d3-56171f237a45 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:42.561474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49bbff-e0e0-4c16-a5ec-ff3805288df4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:42.607221 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:42.631732 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89802626-1380-40b1-bd88-956a73d3f3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:42.632379 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89802626-1380-40b1-bd88-956a73d3f3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:42.632662 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89802626-1380-40b1-bd88-956a73d3f3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:42.632935 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89802626-1380-40b1-bd88-956a73d3f3f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:42.647543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89802626-1380-40b1-bd88-956a73d3f3f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:42.649011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89802626-1380-40b1-bd88-956a73d3f3f3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:42.672663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-89802626-1380-40b1-bd88-956a73d3f3f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 02:59:42.672663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:42.672663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:42.672663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:42.672663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:42.672663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:42.672663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:42.672663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:42.672663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:42.672663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:42.672663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:42.672663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:42.672663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 02:59:42.672663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 02:59:42.672663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 02:59:42.672663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:42.673623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:42.674543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:42.674543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:42.674543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 02:59:42.674543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 02:59:42.674543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 02:59:42.674543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 02:59:42.674543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 02:59:42.674543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 02:59:42.674543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 02:59:42.674543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:42.681773 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 243/497] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Oct 17 02:59:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:59:42.682650 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89802626-1380-40b1-bd88-956a73d3f3f3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:42.730654 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52285bfb-3b9e-46a8-b8d3-56171f237a45 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:42.778263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c27101-540a-44be-bcf1-6aa34430ddb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:42.778926 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c27101-540a-44be-bcf1-6aa34430ddb0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:42.779212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c27101-540a-44be-bcf1-6aa34430ddb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:42.779491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c27101-540a-44be-bcf1-6aa34430ddb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:42.780182 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c27101-540a-44be-bcf1-6aa34430ddb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:42.780488 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c27101-540a-44be-bcf1-6aa34430ddb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:42.781880 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c27101-540a-44be-bcf1-6aa34430ddb0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:42.807135 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 255/498] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Oct 17 02:59:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:42.808177 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c27101-540a-44be-bcf1-6aa34430ddb0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:42.893700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89802626-1380-40b1-bd88-956a73d3f3f3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:42.920828 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:42.932366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67041393-b836-44c9-9bd2-7509b19e2982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:42.932911 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67041393-b836-44c9-9bd2-7509b19e2982 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:42.933189 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67041393-b836-44c9-9bd2-7509b19e2982 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:42.933465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67041393-b836-44c9-9bd2-7509b19e2982 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:42.934197 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67041393-b836-44c9-9bd2-7509b19e2982 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:42.934885 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67041393-b836-44c9-9bd2-7509b19e2982 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:42.936344 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67041393-b836-44c9-9bd2-7509b19e2982 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:42.954978 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 244/499] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Oct 17 02:59:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:42.956280 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bf3a0-0236-4aee-b776-5cdc3086aa47 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:42.956567 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bf3a0-0236-4aee-b776-5cdc3086aa47 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:42.956842 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bf3a0-0236-4aee-b776-5cdc3086aa47 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:42.962750 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 245/500] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:42.962750 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bf3a0-0236-4aee-b776-5cdc3086aa47 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:43.000372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c27101-540a-44be-bcf1-6aa34430ddb0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:43.003503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:43.041161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ec393-9566-4cf8-93e1-f962554bd120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:43.041851 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ec393-9566-4cf8-93e1-f962554bd120 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:43.042240 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ec393-9566-4cf8-93e1-f962554bd120 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:43.042522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ec393-9566-4cf8-93e1-f962554bd120 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:43.054152 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ec393-9566-4cf8-93e1-f962554bd120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:43.055734 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ec393-9566-4cf8-93e1-f962554bd120 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:43.071692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-df8ec393-9566-4cf8-93e1-f962554bd120 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:43.071692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:43.071692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:43.071692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:43.071692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:43.071692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:43.071692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:43.071692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:43.071692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:43.071692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:43.071692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:43.071692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:43.071692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:43.071692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:43.071692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:43.071692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:43.073098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:43.073098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:43.073098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:43.073098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:43.073098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:43.073098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:43.073098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:43.073098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:43.073098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:43.073098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:43.073098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:43.073098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:43.073098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:43.073098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:43.073098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:43.073098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:43.073098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:43.074102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:43.074102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:43.074102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:43.077728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 256/501] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:43.079215 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc5a1d-f0b1-4d90-878b-3965c45a40e1 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:43.079438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bf3a0-0236-4aee-b776-5cdc3086aa47 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:43.079603 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc5a1d-f0b1-4d90-878b-3965c45a40e1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:43.079759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc5a1d-f0b1-4d90-878b-3965c45a40e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:43.099461 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-28fc5a1d-f0b1-4d90-878b-3965c45a40e1 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:43.100657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-28fc5a1d-f0b1-4d90-878b-3965c45a40e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:43.116042 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2daec-56c2-4d59-91c3-90a5526d7319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:43.116803 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2daec-56c2-4d59-91c3-90a5526d7319 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:43.117085 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2daec-56c2-4d59-91c3-90a5526d7319 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:43.117358 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2daec-56c2-4d59-91c3-90a5526d7319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:43.128106 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2daec-56c2-4d59-91c3-90a5526d7319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:43.130725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2daec-56c2-4d59-91c3-90a5526d7319 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:43.147535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-02e2daec-56c2-4d59-91c3-90a5526d7319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:43.147535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:43.147535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:43.147535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:43.147535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:43.147535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:43.147535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:43.147535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:43.147535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:43.147535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:43.147535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:43.147535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:43.147535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:43.147535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:43.147535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:43.147535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:43.149631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:43.149631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:43.149631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:43.149631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:43.149631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:43.149631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:43.149631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:43.149631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:43.149631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:43.149631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:43.149631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:43.149631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:43.149631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:43.149631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:43.149631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:43.149631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:43.149631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:43.151783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:43.151783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:43.151783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:43.151783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 246/502] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:43.151783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2daec-56c2-4d59-91c3-90a5526d7319 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:43.174368 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 257/503] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:43.175387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc5a1d-f0b1-4d90-878b-3965c45a40e1 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:43.190553 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:43.259505 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2daec-56c2-4d59-91c3-90a5526d7319 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:43.294008 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:43.297796 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77a455-e68b-48ea-b8d6-48717a94b777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:43.298537 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77a455-e68b-48ea-b8d6-48717a94b777 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:43.299001 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77a455-e68b-48ea-b8d6-48717a94b777 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:43.299320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77a455-e68b-48ea-b8d6-48717a94b777 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:43.300070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77a455-e68b-48ea-b8d6-48717a94b777 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:43.300471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77a455-e68b-48ea-b8d6-48717a94b777 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:43.302152 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77a455-e68b-48ea-b8d6-48717a94b777 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:43.320369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 247/504] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:43.321612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77a455-e68b-48ea-b8d6-48717a94b777 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:43.323002 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc5a1d-f0b1-4d90-878b-3965c45a40e1 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:43.376926 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388c138-e2d9-4d62-9bce-ba18eb4498f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:43.377565 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388c138-e2d9-4d62-9bce-ba18eb4498f3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:43.377848 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388c138-e2d9-4d62-9bce-ba18eb4498f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:43.378125 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388c138-e2d9-4d62-9bce-ba18eb4498f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:43.378837 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388c138-e2d9-4d62-9bce-ba18eb4498f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:43.388071 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388c138-e2d9-4d62-9bce-ba18eb4498f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:43.389475 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388c138-e2d9-4d62-9bce-ba18eb4498f3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:43.411078 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 258/505] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:43.416315 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388c138-e2d9-4d62-9bce-ba18eb4498f3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:43.421932 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:43.434688 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77a455-e68b-48ea-b8d6-48717a94b777 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:43.500082 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6df525-69b2-47d5-9e32-2e6be5da6936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:43.500699 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6df525-69b2-47d5-9e32-2e6be5da6936 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:43.500983 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6df525-69b2-47d5-9e32-2e6be5da6936 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:43.501258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6df525-69b2-47d5-9e32-2e6be5da6936 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:43.509054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6df525-69b2-47d5-9e32-2e6be5da6936 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:43.510498 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6df525-69b2-47d5-9e32-2e6be5da6936 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:43.522925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-0f6df525-69b2-47d5-9e32-2e6be5da6936 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 02:59:43.522925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:43.522925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:43.522925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:43.522925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:43.522925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:43.522925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:43.522925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:43.522925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:43.522925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:43.522925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:43.522925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:43.522925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 02:59:43.522925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 02:59:43.522925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 02:59:43.522925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:43.523946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:43.524907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:43.524907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:43.524907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 02:59:43.524907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 02:59:43.524907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 02:59:43.524907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 02:59:43.524907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 02:59:43.524907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 02:59:43.524907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 02:59:43.524907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:43.528693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 248/506] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 02:59:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:43.529519 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6df525-69b2-47d5-9e32-2e6be5da6936 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:43.531339 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388c138-e2d9-4d62-9bce-ba18eb4498f3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:43.580132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c71c92-a041-4137-9c5b-4f557965b251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:43.581522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c71c92-a041-4137-9c5b-4f557965b251 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:43.582794 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c71c92-a041-4137-9c5b-4f557965b251 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:43.583544 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c71c92-a041-4137-9c5b-4f557965b251 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:43.593011 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c71c92-a041-4137-9c5b-4f557965b251 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:43.595639 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c71c92-a041-4137-9c5b-4f557965b251 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:43.610797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-86c71c92-a041-4137-9c5b-4f557965b251 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:43.610797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:43.610797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:43.610797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:43.610797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:43.610797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:43.610797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:43.610797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:43.610797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:43.610797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:43.610797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:43.610797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:43.610797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 02:59:43.610797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 02:59:43.610797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 02:59:43.611762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 02:59:43.611762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:43.611762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:43.611762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:43.611762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:43.611762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:43.611762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:43.611762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:43.611762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:43.611762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:43.611762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:43.611762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:43.611762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:43.611762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:43.611762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:43.611762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:43.611762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:43.612661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 02:59:43.612661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 02:59:43.612661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 02:59:43.612661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 02:59:43.612661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 02:59:43.612661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 02:59:43.612661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:43.612661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:43.615906 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 259/507] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:59:43.617308 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c71c92-a041-4137-9c5b-4f557965b251 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:43.622474 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:43.642442 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6df525-69b2-47d5-9e32-2e6be5da6936 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:43.683328 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5dbcd2-b0d8-4099-8f59-4af10f82e871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:43.684487 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5dbcd2-b0d8-4099-8f59-4af10f82e871 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:43.684979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5dbcd2-b0d8-4099-8f59-4af10f82e871 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:43.685403 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5dbcd2-b0d8-4099-8f59-4af10f82e871 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:43.686410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5dbcd2-b0d8-4099-8f59-4af10f82e871 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:43.688326 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5dbcd2-b0d8-4099-8f59-4af10f82e871 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:43.703942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-1a5dbcd2-b0d8-4099-8f59-4af10f82e871 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 02:59:43.703942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:43.703942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 02:59:43.703942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 02:59:43.703942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 02:59:43.703942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 02:59:43.703942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 02:59:43.703942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 02:59:43.703942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 02:59:43.703942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:43.703942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 02:59:43.703942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:43.703942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:43.703942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:43.703942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:43.703942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:43.703942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:43.704998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:43.704998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:43.704998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:43.704998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:43.704998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:43.704998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:43.704998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 02:59:43.704998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 02:59:43.704998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 02:59:43.704998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 02:59:43.704998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:43.704998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:43.704998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:43.704998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:43.704998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:43.704998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:43.705854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:43.705854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:43.705854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:43.705854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:43.705854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:43.705854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:43.705854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:43.705854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:43.705854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:43.705854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:43.705854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 02:59:43.705854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 02:59:43.705854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 02:59:43.705854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:43.706614 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 249/508] 10.222.0.27 () {56 vars in 1129 bytes} [Mon Oct 17 02:59:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:43.708282 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5dbcd2-b0d8-4099-8f59-4af10f82e871 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:43.734691 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:43.758371 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c71c92-a041-4137-9c5b-4f557965b251 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:43.801826 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad6dc5-01b0-45b9-8bdb-2326b724e5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:43.802829 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad6dc5-01b0-45b9-8bdb-2326b724e5b2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:43.803406 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad6dc5-01b0-45b9-8bdb-2326b724e5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:43.803909 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad6dc5-01b0-45b9-8bdb-2326b724e5b2 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:43.804870 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad6dc5-01b0-45b9-8bdb-2326b724e5b2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:43.805392 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad6dc5-01b0-45b9-8bdb-2326b724e5b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:43.807056 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad6dc5-01b0-45b9-8bdb-2326b724e5b2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:43.819046 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5dbcd2-b0d8-4099-8f59-4af10f82e871 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:43.826791 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 260/509] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Oct 17 02:59:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:43.827988 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad6dc5-01b0-45b9-8bdb-2326b724e5b2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:43.861298 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f683d3-6f60-4aac-88be-6bebb09a637f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:43.861922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f683d3-6f60-4aac-88be-6bebb09a637f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:43.862202 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f683d3-6f60-4aac-88be-6bebb09a637f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:43.862475 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f683d3-6f60-4aac-88be-6bebb09a637f None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:43.863276 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f683d3-6f60-4aac-88be-6bebb09a637f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:43.863648 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f683d3-6f60-4aac-88be-6bebb09a637f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:43.865155 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f683d3-6f60-4aac-88be-6bebb09a637f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:43.878618 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 250/510] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 02:59:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:43.881038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f683d3-6f60-4aac-88be-6bebb09a637f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:43.898174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:43.940227 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad6dc5-01b0-45b9-8bdb-2326b724e5b2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:43.969471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:43.978312 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970fd5ba-d6ba-4afc-8673-948c92e28950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:43.979286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fd5ba-d6ba-4afc-8673-948c92e28950 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:43.979826 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fd5ba-d6ba-4afc-8673-948c92e28950 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:43.980313 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fd5ba-d6ba-4afc-8673-948c92e28950 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:43.981154 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970fd5ba-d6ba-4afc-8673-948c92e28950 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:43.981693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970fd5ba-d6ba-4afc-8673-948c92e28950 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:43.983302 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970fd5ba-d6ba-4afc-8673-948c92e28950 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:43.988577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 261/511] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Oct 17 02:59:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:43.989981 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970fd5ba-d6ba-4afc-8673-948c92e28950 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:43.991300 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f683d3-6f60-4aac-88be-6bebb09a637f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:44.028378 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f576c8-46cd-48ac-bb63-9af01f1e1a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:44.028766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f576c8-46cd-48ac-bb63-9af01f1e1a86 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:44.029090 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f576c8-46cd-48ac-bb63-9af01f1e1a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:44.029370 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f576c8-46cd-48ac-bb63-9af01f1e1a86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:44.097573 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970fd5ba-d6ba-4afc-8673-948c92e28950 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:44.102366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f576c8-46cd-48ac-bb63-9af01f1e1a86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:44.103884 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f576c8-46cd-48ac-bb63-9af01f1e1a86 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:44.128971 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:44.140889 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab456c1c-bdf7-4065-846e-b46d5eb309ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:44.142092 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab456c1c-bdf7-4065-846e-b46d5eb309ad None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:44.142618 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab456c1c-bdf7-4065-846e-b46d5eb309ad None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:44.143162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab456c1c-bdf7-4065-846e-b46d5eb309ad None admin] PATH_INFO: `/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/01637fc07ba846aebcb42325bed46216/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:44.146009 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab456c1c-bdf7-4065-846e-b46d5eb309ad None admin] RBAC: Authorizing `identity:create_grant(project_id=89b0ee95955f465eb9ee5b8ae337d9c2, user_id=01637fc07ba846aebcb42325bed46216, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:44.147820 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab456c1c-bdf7-4065-846e-b46d5eb309ad None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:44.163879 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 262/512] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 02:59:43 2022] PUT /identity/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/01637fc07ba846aebcb42325bed46216/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:59:44.165254 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab456c1c-bdf7-4065-846e-b46d5eb309ad None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:44.201301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:44.230619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 251/513] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:59:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 02:59:44.232204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f576c8-46cd-48ac-bb63-9af01f1e1a86 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:44.288390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab456c1c-bdf7-4065-846e-b46d5eb309ad None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:44.329979 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2d92d-5d71-4d80-9af9-8044bebcfe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:44.330906 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2d92d-5d71-4d80-9af9-8044bebcfe2d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:44.331269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2d92d-5d71-4d80-9af9-8044bebcfe2d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:44.331656 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2d92d-5d71-4d80-9af9-8044bebcfe2d None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:44.333383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2d92d-5d71-4d80-9af9-8044bebcfe2d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:44.333758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2d92d-5d71-4d80-9af9-8044bebcfe2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:44.335341 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2d92d-5d71-4d80-9af9-8044bebcfe2d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:44.341992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 263/514] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Oct 17 02:59:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:44.343521 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1524934-d4c1-4f85-a103-001b66c31701 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:44.343923 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1524934-d4c1-4f85-a103-001b66c31701 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:44.344312 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1524934-d4c1-4f85-a103-001b66c31701 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:44.346173 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 264/515] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:44.354729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548c1aa-33a0-4715-bb51-713c0bb33b00 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:44.355190 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548c1aa-33a0-4715-bb51-713c0bb33b00 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:44.355576 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548c1aa-33a0-4715-bb51-713c0bb33b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:44.357009 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f576c8-46cd-48ac-bb63-9af01f1e1a86 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:44.396356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8548c1aa-33a0-4715-bb51-713c0bb33b00 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:44.397846 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8548c1aa-33a0-4715-bb51-713c0bb33b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:44.401397 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bffdc6-69cb-4c0a-9275-73dcafefa7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:44.402437 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bffdc6-69cb-4c0a-9275-73dcafefa7cf None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:44.402993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bffdc6-69cb-4c0a-9275-73dcafefa7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:44.403525 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bffdc6-69cb-4c0a-9275-73dcafefa7cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:44.422651 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bffdc6-69cb-4c0a-9275-73dcafefa7cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:44.424167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bffdc6-69cb-4c0a-9275-73dcafefa7cf None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:44.444818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-59bffdc6-69cb-4c0a-9275-73dcafefa7cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:44.444818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:44.444818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:44.444818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:44.444818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:44.444818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:44.444818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:44.444818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:44.444818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:44.444818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:44.444818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:44.444818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:44.444818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:44.444818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:44.444818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:44.444818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:44.445846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:44.445846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:44.445846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:44.445846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:44.445846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:44.445846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:44.445846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:44.445846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:44.445846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:44.445846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:44.445846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:44.445846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:44.445846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:44.445846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:44.445846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:44.445846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:44.445846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:44.447877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:44.447877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:44.447877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:44.447877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 252/516] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:44.447877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba21aa7-90a7-4676-b0be-b5b37dbb9bea None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:44.447877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba21aa7-90a7-4676-b0be-b5b37dbb9bea None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:44.447877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba21aa7-90a7-4676-b0be-b5b37dbb9bea None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:44.449344 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 253/517] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:44.456447 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fc77b-dc56-4261-ab93-f098a24ebe59 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:44.456728 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fc77b-dc56-4261-ab93-f098a24ebe59 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:44.456999 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fc77b-dc56-4261-ab93-f098a24ebe59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:44.467783 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:44.478046 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-382fc77b-dc56-4261-ab93-f098a24ebe59 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:44.479371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-382fc77b-dc56-4261-ab93-f098a24ebe59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:44.507064 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 265/518] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:44.508040 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548c1aa-33a0-4715-bb51-713c0bb33b00 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:44.528334 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:44.611460 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 254/519] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:44.643976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548c1aa-33a0-4715-bb51-713c0bb33b00 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:44.680872 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09a6bd-d4c1-4902-a194-16cc47bef81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:44.681537 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09a6bd-d4c1-4902-a194-16cc47bef81b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:44.681782 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09a6bd-d4c1-4902-a194-16cc47bef81b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:44.682058 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09a6bd-d4c1-4902-a194-16cc47bef81b None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:44.682705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09a6bd-d4c1-4902-a194-16cc47bef81b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:44.683076 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09a6bd-d4c1-4902-a194-16cc47bef81b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:44.684466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09a6bd-d4c1-4902-a194-16cc47bef81b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:44.697532 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 266/520] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:44.702978 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fc77b-dc56-4261-ab93-f098a24ebe59 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:44.819153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fc77b-dc56-4261-ab93-f098a24ebe59 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:44.821421 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09a6bd-d4c1-4902-a194-16cc47bef81b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:44.871622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:44.883815 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504de32-dc13-4ae9-a20b-55fb5e6e6e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:44.885058 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504de32-dc13-4ae9-a20b-55fb5e6e6e5d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:44.885342 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504de32-dc13-4ae9-a20b-55fb5e6e6e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:44.885749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504de32-dc13-4ae9-a20b-55fb5e6e6e5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:44.901242 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2504de32-dc13-4ae9-a20b-55fb5e6e6e5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:44.902724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2504de32-dc13-4ae9-a20b-55fb5e6e6e5d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:44.921993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2504de32-dc13-4ae9-a20b-55fb5e6e6e5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:44.921993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:44.921993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:44.921993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:44.921993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:44.921993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:44.921993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:44.921993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:44.921993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:44.921993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:44.921993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:44.921993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:44.921993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 02:59:44.921993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 02:59:44.921993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 02:59:44.922940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 02:59:44.922940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:44.922940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:44.922940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:44.922940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:44.922940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:44.922940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:44.922940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:44.922940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:44.922940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:44.922940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:44.922940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:44.922940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:44.922940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:44.922940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:44.922940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:44.922940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:44.924712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 02:59:44.924712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 02:59:44.924712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 02:59:44.924712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 02:59:44.924712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 02:59:44.924712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 02:59:44.924712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:44.924712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:44.925773 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 255/521] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:44.933649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504de32-dc13-4ae9-a20b-55fb5e6e6e5d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:44.978233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09a6bd-d4c1-4902-a194-16cc47bef81b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:44.988043 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:45.022630 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f0efb-1c5a-41d5-bba6-6ded81f49b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:45.023360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f0efb-1c5a-41d5-bba6-6ded81f49b7a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:45.023643 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f0efb-1c5a-41d5-bba6-6ded81f49b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:45.023905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f0efb-1c5a-41d5-bba6-6ded81f49b7a None admin] PATH_INFO: `/v3/services/network` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:45.024548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f0efb-1c5a-41d5-bba6-6ded81f49b7a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:45.025927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f0efb-1c5a-41d5-bba6-6ded81f49b7a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:45.037178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-900f0efb-1c5a-41d5-bba6-6ded81f49b7a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 02:59:45.037178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:45.037178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 02:59:45.037178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 02:59:45.037178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 02:59:45.037178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 02:59:45.037178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 02:59:45.037178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 02:59:45.037178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 02:59:45.037178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:45.037178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 02:59:45.037178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:45.037178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:45.037178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:45.037178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:45.037178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:45.037178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:45.039622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:45.039622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:45.039622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:45.039622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:45.039622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:45.039622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:45.039622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 02:59:45.039622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 02:59:45.039622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 02:59:45.039622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 02:59:45.039622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:45.039622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:45.039622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:45.039622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:45.039622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:45.039622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:45.040788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:45.040788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:45.040788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:45.040788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:45.040788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:45.040788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:45.040788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:45.040788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:45.040788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:45.040788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:45.040788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 02:59:45.040788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 02:59:45.040788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 02:59:45.040788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:45.040788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 267/522] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:44 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:59:45.041790 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f0efb-1c5a-41d5-bba6-6ded81f49b7a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:45.085334 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504de32-dc13-4ae9-a20b-55fb5e6e6e5d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:45.122922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97084fd9-8226-4498-bb56-f749f1bebcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:45.123467 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97084fd9-8226-4498-bb56-f749f1bebcfe None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:45.123741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97084fd9-8226-4498-bb56-f749f1bebcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:45.124153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97084fd9-8226-4498-bb56-f749f1bebcfe None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:45.124848 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97084fd9-8226-4498-bb56-f749f1bebcfe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:45.125154 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97084fd9-8226-4498-bb56-f749f1bebcfe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:45.126702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97084fd9-8226-4498-bb56-f749f1bebcfe None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:45.143286 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 256/523] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 02:59:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:45.144499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97084fd9-8226-4498-bb56-f749f1bebcfe None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:45.157347 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f0efb-1c5a-41d5-bba6-6ded81f49b7a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:45.161737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:45.201439 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1bc6e-85d5-4167-8fa2-e27ff5f45bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:45.202079 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1bc6e-85d5-4167-8fa2-e27ff5f45bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:45.202361 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1bc6e-85d5-4167-8fa2-e27ff5f45bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:45.202637 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1bc6e-85d5-4167-8fa2-e27ff5f45bd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:45.217103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1bc6e-85d5-4167-8fa2-e27ff5f45bd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:45.218688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1bc6e-85d5-4167-8fa2-e27ff5f45bd6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:45.234800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-61a1bc6e-85d5-4167-8fa2-e27ff5f45bd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:45.234800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:45.234800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:45.234800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:45.234800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:45.234800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:45.234800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:45.234800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:45.234800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:45.234800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:45.234800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:45.234800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:45.234800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:45.234800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:45.234800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:45.234800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:45.235813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:45.235813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:45.235813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:45.235813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:45.235813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:45.235813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:45.235813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:45.235813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:45.235813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:45.235813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:45.235813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:45.235813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:45.235813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:45.235813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:45.235813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:45.235813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:45.235813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:45.236792 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:45.236792 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:45.236792 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:45.241527 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 268/524] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:45.242546 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1bc6e-85d5-4167-8fa2-e27ff5f45bd6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:45.248819 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:45.268980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97084fd9-8226-4498-bb56-f749f1bebcfe None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:45.309643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301cb14-5bab-4817-8f94-997efe1c9814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:45.310574 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301cb14-5bab-4817-8f94-997efe1c9814 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:45.311026 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301cb14-5bab-4817-8f94-997efe1c9814 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:45.311438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301cb14-5bab-4817-8f94-997efe1c9814 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:45.312343 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301cb14-5bab-4817-8f94-997efe1c9814 None admin] RBAC: Adding query filter params (name=network) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:45.312880 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301cb14-5bab-4817-8f94-997efe1c9814 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:45.314424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301cb14-5bab-4817-8f94-997efe1c9814 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:45.324668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 257/525] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Oct 17 02:59:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:45.325707 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301cb14-5bab-4817-8f94-997efe1c9814 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:45.356974 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1bc6e-85d5-4167-8fa2-e27ff5f45bd6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:45.394935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41f770-5409-415f-b8a3-670e46dfbb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:45.395498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41f770-5409-415f-b8a3-670e46dfbb21 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:45.395781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41f770-5409-415f-b8a3-670e46dfbb21 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:45.396116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41f770-5409-415f-b8a3-670e46dfbb21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:45.396786 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41f770-5409-415f-b8a3-670e46dfbb21 None admin] RBAC: Adding query filter params (role.id=78dc08989b32455389b21096f5d57e17, scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2, user.id=eceda8b3a8ba4233851d0138d8f4d06a) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:45.397106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41f770-5409-415f-b8a3-670e46dfbb21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:45.398588 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41f770-5409-415f-b8a3-670e46dfbb21 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:45.405277 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 269/526] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 02:59:45 2022] GET /identity/v3/role_assignments?user.id=eceda8b3a8ba4233851d0138d8f4d06a&scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2&role.id=78dc08989b32455389b21096f5d57e17 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:45.406144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41f770-5409-415f-b8a3-670e46dfbb21 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:45.438987 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301cb14-5bab-4817-8f94-997efe1c9814 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:45.450101 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:45.534028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13b7c6-c5c8-4a15-b9a5-554355b99aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:45.534655 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13b7c6-c5c8-4a15-b9a5-554355b99aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:45.534971 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13b7c6-c5c8-4a15-b9a5-554355b99aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:45.536072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13b7c6-c5c8-4a15-b9a5-554355b99aa9 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:45.536472 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13b7c6-c5c8-4a15-b9a5-554355b99aa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:45.536741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13b7c6-c5c8-4a15-b9a5-554355b99aa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:45.538158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13b7c6-c5c8-4a15-b9a5-554355b99aa9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:45.544056 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41f770-5409-415f-b8a3-670e46dfbb21 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:45.564575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 258/527] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:45.570824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13b7c6-c5c8-4a15-b9a5-554355b99aa9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:45.589521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:45.618616 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21036737-6e74-4603-a19c-c6c51ffe7146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:45.619556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21036737-6e74-4603-a19c-c6c51ffe7146 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:45.620115 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21036737-6e74-4603-a19c-c6c51ffe7146 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:45.620572 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21036737-6e74-4603-a19c-c6c51ffe7146 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:45.621463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21036737-6e74-4603-a19c-c6c51ffe7146 None admin] RBAC: Adding query filter params (type=network) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:45.621948 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21036737-6e74-4603-a19c-c6c51ffe7146 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:45.623528 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21036737-6e74-4603-a19c-c6c51ffe7146 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:45.629838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 270/528] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Oct 17 02:59:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:45.694977 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13b7c6-c5c8-4a15-b9a5-554355b99aa9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:45.732919 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e320ea-53c9-4bc9-b503-2e5c99e6b811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:45.733552 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e320ea-53c9-4bc9-b503-2e5c99e6b811 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:45.733831 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e320ea-53c9-4bc9-b503-2e5c99e6b811 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:45.734104 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e320ea-53c9-4bc9-b503-2e5c99e6b811 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:45.734778 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e320ea-53c9-4bc9-b503-2e5c99e6b811 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:45.736278 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e320ea-53c9-4bc9-b503-2e5c99e6b811 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:45.768318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 259/529] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 02:59:45 2022] POST /identity/v3/users => generated 254 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 02:59:46.761867 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ba92a-7143-4334-bbac-683eaca59736 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:46.762590 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ba92a-7143-4334-bbac-683eaca59736 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:46.762590 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ba92a-7143-4334-bbac-683eaca59736 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:46.764181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 271/530] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:46.770135 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4629d8-98bd-48fe-a0b5-86dbe787f2bc None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:46.770673 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4629d8-98bd-48fe-a0b5-86dbe787f2bc None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:46.770673 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4629d8-98bd-48fe-a0b5-86dbe787f2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:46.794764 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1e4629d8-98bd-48fe-a0b5-86dbe787f2bc None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:46.796043 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1e4629d8-98bd-48fe-a0b5-86dbe787f2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:46.877971 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:46.888016 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 260/531] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:46.910972 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced6e66-2ad3-4abd-abbf-e9b501d16195 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:46.911276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced6e66-2ad3-4abd-abbf-e9b501d16195 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:46.911535 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced6e66-2ad3-4abd-abbf-e9b501d16195 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:46.913263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 272/532] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:46.918538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df3867-d381-40fe-9297-c1b8db3677d1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:46.918836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df3867-d381-40fe-9297-c1b8db3677d1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:46.919195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df3867-d381-40fe-9297-c1b8db3677d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:46.945178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c8df3867-d381-40fe-9297-c1b8db3677d1 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:46.946427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c8df3867-d381-40fe-9297-c1b8db3677d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:47.030127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 261/533] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:47.189978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced6e66-2ad3-4abd-abbf-e9b501d16195 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:47.294424 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:47.312640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced6e66-2ad3-4abd-abbf-e9b501d16195 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:47.361201 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4820b-58fc-4be9-abeb-26d12f1b61b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:47.361833 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4820b-58fc-4be9-abeb-26d12f1b61b2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:47.362122 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4820b-58fc-4be9-abeb-26d12f1b61b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:47.362402 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4820b-58fc-4be9-abeb-26d12f1b61b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:47.383751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4820b-58fc-4be9-abeb-26d12f1b61b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:47.388581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-dac4820b-58fc-4be9-abeb-26d12f1b61b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:59:47.388581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:47.388581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:47.388581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:47.388581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:47.388581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:47.388581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:47.388581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:47.388581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:47.388581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:47.388581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:47.388581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:47.388581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:47.388581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:47.388581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:47.388581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:47.389483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:47.389483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:47.389483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:47.389483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:47.389483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:47.389483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:47.389483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:47.389483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:47.389483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:47.389483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:47.389483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:47.389483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:47.389483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:47.389483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:47.389483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:47.389483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:47.390337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:47.390337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:47.390337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:47.390337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:47.390337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:47.390337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:47.390337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:47.390337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:47.390337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:47.390337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:47.390337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:59:47.390337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:47.390337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 273/534] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 02:59:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:47.396198 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df3867-d381-40fe-9297-c1b8db3677d1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:47.409568 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4820b-58fc-4be9-abeb-26d12f1b61b2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:47.563959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4820b-58fc-4be9-abeb-26d12f1b61b2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:47.576193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df3867-d381-40fe-9297-c1b8db3677d1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:47.599547 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4e6ce-5452-4dba-b163-a40af5091388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:47.600158 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4e6ce-5452-4dba-b163-a40af5091388 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:47.600440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4e6ce-5452-4dba-b163-a40af5091388 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:47.600712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4e6ce-5452-4dba-b163-a40af5091388 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:47.601379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4e6ce-5452-4dba-b163-a40af5091388 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:47.602743 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4e6ce-5452-4dba-b163-a40af5091388 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:47.624963 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:47.630941 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 274/535] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Oct 17 02:59:47 2022] POST /identity/v3/services => generated 251 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 02:59:47.647201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9967e-41fd-434d-9e76-e9f8f4c3fb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:47.647837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9967e-41fd-434d-9e76-e9f8f4c3fb72 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:47.648122 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9967e-41fd-434d-9e76-e9f8f4c3fb72 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:47.648400 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9967e-41fd-434d-9e76-e9f8f4c3fb72 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:47.649099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9967e-41fd-434d-9e76-e9f8f4c3fb72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:47.649413 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9967e-41fd-434d-9e76-e9f8f4c3fb72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:47.650958 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9967e-41fd-434d-9e76-e9f8f4c3fb72 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:47.670881 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 262/536] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:47.678049 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4e6ce-5452-4dba-b163-a40af5091388 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:47.683278 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:47.806027 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4e6ce-5452-4dba-b163-a40af5091388 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:47.868794 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f4acf-ffb7-4be8-bb0b-837d45365ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:47.869367 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f4acf-ffb7-4be8-bb0b-837d45365ccb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:47.869648 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f4acf-ffb7-4be8-bb0b-837d45365ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:47.869933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f4acf-ffb7-4be8-bb0b-837d45365ccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:47.884466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f4acf-ffb7-4be8-bb0b-837d45365ccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:47.886378 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f4acf-ffb7-4be8-bb0b-837d45365ccb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:47.904944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-985f4acf-ffb7-4be8-bb0b-837d45365ccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:47.904944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:47.904944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:47.904944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:47.904944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:47.904944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:47.904944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:47.904944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:47.904944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:47.904944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:47.904944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:47.904944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:47.904944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:47.904944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:47.904944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:47.904944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:47.905920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:47.905920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:47.905920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:47.905920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:47.905920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:47.905920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:47.905920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:47.905920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:47.905920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:47.905920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:47.905920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:47.905920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:47.905920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:47.905920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:47.905920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:47.905920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:47.905920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:47.906797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:47.906797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:47.906797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:47.909633 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 275/537] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:47.914982 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9967e-41fd-434d-9e76-e9f8f4c3fb72 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:48.027116 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9967e-41fd-434d-9e76-e9f8f4c3fb72 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:48.065559 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e8380-81ad-4eaf-aa80-a84644a2ffd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:48.066168 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e8380-81ad-4eaf-aa80-a84644a2ffd9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:48.066440 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e8380-81ad-4eaf-aa80-a84644a2ffd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:48.067018 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e8380-81ad-4eaf-aa80-a84644a2ffd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:48.067677 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e8380-81ad-4eaf-aa80-a84644a2ffd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:48.067977 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e8380-81ad-4eaf-aa80-a84644a2ffd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:48.069446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e8380-81ad-4eaf-aa80-a84644a2ffd9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:48.083862 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 263/538] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:48.089282 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f4acf-ffb7-4be8-bb0b-837d45365ccb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:48.163750 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0833456-7a9f-4c37-b961-b5f1d34867b3 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:48.163939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0833456-7a9f-4c37-b961-b5f1d34867b3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:48.164207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0833456-7a9f-4c37-b961-b5f1d34867b3 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:48.165811 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 264/539] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 02:59:48.171525 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422949f9-eeda-495e-b853-117358bd6b53 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:48.171803 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422949f9-eeda-495e-b853-117358bd6b53 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:48.172081 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422949f9-eeda-495e-b853-117358bd6b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:48.195231 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:48.203260 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-422949f9-eeda-495e-b853-117358bd6b53 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:48.204455 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-422949f9-eeda-495e-b853-117358bd6b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:48.249416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f4acf-ffb7-4be8-bb0b-837d45365ccb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:48.284969 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:48.327353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40e430-2aad-40b0-8f4d-a8c6ac40507f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:48.328003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40e430-2aad-40b0-8f4d-a8c6ac40507f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:48.328277 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40e430-2aad-40b0-8f4d-a8c6ac40507f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:48.329507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40e430-2aad-40b0-8f4d-a8c6ac40507f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:48.338759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40e430-2aad-40b0-8f4d-a8c6ac40507f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:48.340272 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40e430-2aad-40b0-8f4d-a8c6ac40507f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:48.345940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 265/540] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:59:48.362157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-6f40e430-2aad-40b0-8f4d-a8c6ac40507f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 02:59:48.362157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:48.362157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:48.362157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:48.362157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:48.362157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:48.362157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:48.362157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:48.362157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:48.362157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:48.362157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:48.362157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:48.362157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 02:59:48.362157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 02:59:48.362157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 02:59:48.362157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:48.363200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:48.364585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:48.364585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:48.364585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 02:59:48.364585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 02:59:48.364585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 02:59:48.364585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 02:59:48.364585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 02:59:48.364585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 02:59:48.364585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 02:59:48.364585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:48.364585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 276/541] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Oct 17 02:59:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:48.369256 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422949f9-eeda-495e-b853-117358bd6b53 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:48.451194 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed5af0-e6d6-4721-9e88-2ee765e210bf None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:48.451879 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed5af0-e6d6-4721-9e88-2ee765e210bf None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:48.452369 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed5af0-e6d6-4721-9e88-2ee765e210bf None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:48.454169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 277/542] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:48.459470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be839d7a-a25f-4c4b-bc63-59ddef0212de None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:48.460013 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be839d7a-a25f-4c4b-bc63-59ddef0212de None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:48.460526 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be839d7a-a25f-4c4b-bc63-59ddef0212de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:48.493377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-be839d7a-a25f-4c4b-bc63-59ddef0212de None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:48.495003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-be839d7a-a25f-4c4b-bc63-59ddef0212de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:48.532712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422949f9-eeda-495e-b853-117358bd6b53 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:48.600010 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3870f-3072-44a0-9089-f869557e9b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:48.601828 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3870f-3072-44a0-9089-f869557e9b0d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:48.602489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3870f-3072-44a0-9089-f869557e9b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:48.604013 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3870f-3072-44a0-9089-f869557e9b0d None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:48.604980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3870f-3072-44a0-9089-f869557e9b0d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:48.605686 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3870f-3072-44a0-9089-f869557e9b0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:48.608149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3870f-3072-44a0-9089-f869557e9b0d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:48.628318 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 278/543] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 02:59:48.647929 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 266/544] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Oct 17 02:59:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:48.649041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be839d7a-a25f-4c4b-bc63-59ddef0212de None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:48.653097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3870f-3072-44a0-9089-f869557e9b0d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:48.722165 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:48.731119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:48.818159 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be839d7a-a25f-4c4b-bc63-59ddef0212de None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:48.844691 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3870f-3072-44a0-9089-f869557e9b0d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:48.888845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea33ce-ff18-4731-9402-7c5081b40bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:48.889452 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99f1ab-f038-439a-823c-bc9ca3b9dcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:48.889601 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea33ce-ff18-4731-9402-7c5081b40bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:48.889807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea33ce-ff18-4731-9402-7c5081b40bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:48.890012 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea33ce-ff18-4731-9402-7c5081b40bd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:48.890109 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99f1ab-f038-439a-823c-bc9ca3b9dcf3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:48.890379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99f1ab-f038-439a-823c-bc9ca3b9dcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:48.890773 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99f1ab-f038-439a-823c-bc9ca3b9dcf3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:48.905851 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99f1ab-f038-439a-823c-bc9ca3b9dcf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:48.913193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-3d99f1ab-f038-439a-823c-bc9ca3b9dcf3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:59:48.913193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:48.913193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:48.913193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:48.913193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:48.913193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:48.913193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:48.913193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:48.913193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:48.913193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:48.913193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:48.913193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:48.913193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:48.913193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:48.913193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:48.913193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:48.914132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:48.914132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:48.914132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:48.914132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:48.914132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:48.914132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:48.914132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:48.914132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:48.914132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:48.914132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:48.914132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:48.914132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:48.914132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:48.914132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:48.914132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:48.914132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:48.915007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:48.915007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:48.915007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:48.915007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:48.915007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:48.915007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:48.915007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:48.915007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:48.915007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:48.915007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:48.915007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:59:48.915007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:48.915007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 279/545] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 02:59:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:59:48.919133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea373d93-1c25-4fdd-8232-31a85e4e2716 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:48.919133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea373d93-1c25-4fdd-8232-31a85e4e2716 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:48.919133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea373d93-1c25-4fdd-8232-31a85e4e2716 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:48.920523 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 280/546] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:48.921259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea373d93-1c25-4fdd-8232-31a85e4e2716 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:48.977229 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea33ce-ff18-4731-9402-7c5081b40bd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:48.978621 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea33ce-ff18-4731-9402-7c5081b40bd4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:49.008932 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:49.029483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea373d93-1c25-4fdd-8232-31a85e4e2716 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:49.038628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:49.068547 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3399b4-eec1-4287-9059-d290f026739c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:49.069267 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3399b4-eec1-4287-9059-d290f026739c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:49.069549 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3399b4-eec1-4287-9059-d290f026739c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:49.069924 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3399b4-eec1-4287-9059-d290f026739c None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:49.070585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3399b4-eec1-4287-9059-d290f026739c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:49.070917 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3399b4-eec1-4287-9059-d290f026739c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:49.072393 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3399b4-eec1-4287-9059-d290f026739c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:49.079392 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 267/547] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:59:48 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 02:59:49.085248 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 281/548] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:49.085248 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3399b4-eec1-4287-9059-d290f026739c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:49.085443 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09d686-e1bc-46d3-92a2-34a72d3f7577 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:49.085443 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09d686-e1bc-46d3-92a2-34a72d3f7577 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:49.085443 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09d686-e1bc-46d3-92a2-34a72d3f7577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:49.103595 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-fe09d686-e1bc-46d3-92a2-34a72d3f7577 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:49.104780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fe09d686-e1bc-46d3-92a2-34a72d3f7577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:49.174135 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 268/549] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:49 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 02:59:49.174997 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09d686-e1bc-46d3-92a2-34a72d3f7577 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:49.204916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3399b4-eec1-4287-9059-d290f026739c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:49.245888 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca4d8c-2adc-4bd6-81f8-89cbb5c0f95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:49.246766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca4d8c-2adc-4bd6-81f8-89cbb5c0f95f None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:49.247074 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca4d8c-2adc-4bd6-81f8-89cbb5c0f95f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:49.247352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca4d8c-2adc-4bd6-81f8-89cbb5c0f95f None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:49.248014 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca4d8c-2adc-4bd6-81f8-89cbb5c0f95f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:49.249761 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca4d8c-2adc-4bd6-81f8-89cbb5c0f95f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:49.252523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:49.273464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 282/550] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Oct 17 02:59:49 2022] POST /identity/v3/services => generated 240 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:59:49.274482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca4d8c-2adc-4bd6-81f8-89cbb5c0f95f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:49.310263 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09d686-e1bc-46d3-92a2-34a72d3f7577 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:49.318569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:49.391550 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de638ee3-f381-4014-94f3-58292f463769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:49.392649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de638ee3-f381-4014-94f3-58292f463769 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:49.394086 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de638ee3-f381-4014-94f3-58292f463769 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:49.395954 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de638ee3-f381-4014-94f3-58292f463769 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:49.419330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de638ee3-f381-4014-94f3-58292f463769 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:49.421048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de638ee3-f381-4014-94f3-58292f463769 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:49.455036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-de638ee3-f381-4014-94f3-58292f463769 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:49.455036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:49.455036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:49.455036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:49.455036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:49.455036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:49.455036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:49.455036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:49.455036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:49.455036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:49.455036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:49.455036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:49.455036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:49.455036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:49.455036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:49.455036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:49.456352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:49.456352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:49.456352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:49.456352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:49.456352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:49.456352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:49.456352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:49.456352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:49.456352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:49.456352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:49.456352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:49.456352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:49.456352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:49.456352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:49.456352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:49.456352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:49.456352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:49.458189 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:49.458189 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:49.458189 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:49.458189 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 269/551] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:49.466477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca4d8c-2adc-4bd6-81f8-89cbb5c0f95f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:49.467696 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de638ee3-f381-4014-94f3-58292f463769 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:49.530453 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c8ee7-4b1e-49ea-9a42-2b33e8a2cc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:49.531094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c8ee7-4b1e-49ea-9a42-2b33e8a2cc49 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:49.531376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c8ee7-4b1e-49ea-9a42-2b33e8a2cc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:49.531646 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c8ee7-4b1e-49ea-9a42-2b33e8a2cc49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:49.549490 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c8ee7-4b1e-49ea-9a42-2b33e8a2cc49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:49.551511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c8ee7-4b1e-49ea-9a42-2b33e8a2cc49 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:49.568718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-6d9c8ee7-4b1e-49ea-9a42-2b33e8a2cc49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:49.568718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:49.568718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:49.568718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:49.568718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:49.568718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:49.568718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:49.568718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:49.568718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:49.568718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:49.568718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:49.568718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:49.568718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:49.568718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:49.568718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:49.568718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:49.569697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:49.569697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:49.569697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:49.569697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:49.569697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:49.569697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:49.569697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:49.569697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:49.569697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:49.569697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:49.569697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:49.569697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:49.569697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:49.569697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:49.569697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:49.569697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:49.569697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:49.570586 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:49.570586 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:49.570586 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:49.570586 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 283/552] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:49.570815 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c8ee7-4b1e-49ea-9a42-2b33e8a2cc49 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:49.620618 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de638ee3-f381-4014-94f3-58292f463769 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:49.633808 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:49.662878 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:49.684757 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452a6a2-f4bf-4165-8181-371c063c5734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:49.685372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452a6a2-f4bf-4165-8181-371c063c5734 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:49.685651 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452a6a2-f4bf-4165-8181-371c063c5734 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:49.685925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452a6a2-f4bf-4165-8181-371c063c5734 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:49.686577 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452a6a2-f4bf-4165-8181-371c063c5734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:49.690921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452a6a2-f4bf-4165-8181-371c063c5734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:49.690921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452a6a2-f4bf-4165-8181-371c063c5734 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:49.698601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c8ee7-4b1e-49ea-9a42-2b33e8a2cc49 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:49.708462 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 270/553] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:49.709391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452a6a2-f4bf-4165-8181-371c063c5734 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:49.737571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca924a9d-7221-4822-83b3-2ddc544be9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:49.739937 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca924a9d-7221-4822-83b3-2ddc544be9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:49.740436 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca924a9d-7221-4822-83b3-2ddc544be9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:49.740916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca924a9d-7221-4822-83b3-2ddc544be9f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:49.753091 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca924a9d-7221-4822-83b3-2ddc544be9f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:49.770693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ca924a9d-7221-4822-83b3-2ddc544be9f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:49.770693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:49.770693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:49.770693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:49.770693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:49.770693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:49.770693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:49.770693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:49.770693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:49.770693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:49.770693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:49.770693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:49.770693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:49.770693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:49.770693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:49.770693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:49.771759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:49.771759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:49.771759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:49.771759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:49.771759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:49.771759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:49.771759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:49.771759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:49.771759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:49.771759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:49.771759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:49.771759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:49.771759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:49.771759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:49.771759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:49.771759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:49.772625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:49.772625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:49.772625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:49.772625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:49.772625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:49.772625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:49.772625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:49.772625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:49.772625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:49.772625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:49.772625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:49.772625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:49.777919 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 284/554] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Oct 17 02:59:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:49.779002 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca924a9d-7221-4822-83b3-2ddc544be9f3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:49.826441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452a6a2-f4bf-4165-8181-371c063c5734 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:49.862112 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47397cfc-793c-43ad-a3b9-1ce2ec48a386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:49.862724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47397cfc-793c-43ad-a3b9-1ce2ec48a386 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:49.863035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47397cfc-793c-43ad-a3b9-1ce2ec48a386 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:49.863310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47397cfc-793c-43ad-a3b9-1ce2ec48a386 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:49.864245 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47397cfc-793c-43ad-a3b9-1ce2ec48a386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:49.864547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47397cfc-793c-43ad-a3b9-1ce2ec48a386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:49.866075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47397cfc-793c-43ad-a3b9-1ce2ec48a386 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:49.883651 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 271/555] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:49.884655 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47397cfc-793c-43ad-a3b9-1ce2ec48a386 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:49.915049 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:49.993945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47397cfc-793c-43ad-a3b9-1ce2ec48a386 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:49.996612 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca924a9d-7221-4822-83b3-2ddc544be9f3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:50.029396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54498cfc-7bb6-4438-aad6-729513bd6f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:50.030030 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54498cfc-7bb6-4438-aad6-729513bd6f5b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:50.030308 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54498cfc-7bb6-4438-aad6-729513bd6f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:50.030576 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54498cfc-7bb6-4438-aad6-729513bd6f5b None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:50.031343 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54498cfc-7bb6-4438-aad6-729513bd6f5b None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:50.031630 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54498cfc-7bb6-4438-aad6-729513bd6f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:50.032985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54498cfc-7bb6-4438-aad6-729513bd6f5b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:50.046924 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 272/556] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Oct 17 02:59:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:50.047825 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54498cfc-7bb6-4438-aad6-729513bd6f5b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:50.065973 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b3d08-1508-44be-9944-31a4a2b82ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:50.068142 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b3d08-1508-44be-9944-31a4a2b82ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:50.068669 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b3d08-1508-44be-9944-31a4a2b82ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:50.069255 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b3d08-1508-44be-9944-31a4a2b82ec0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:50.079722 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98b3d08-1508-44be-9944-31a4a2b82ec0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:50.081824 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98b3d08-1508-44be-9944-31a4a2b82ec0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:50.086795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:50.101245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e98b3d08-1508-44be-9944-31a4a2b82ec0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:50.101245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:50.101245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:50.101245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:50.101245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:50.101245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:50.101245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:50.101245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:50.101245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:50.101245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:50.101245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:50.101245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:50.101245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 02:59:50.101245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 02:59:50.101245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 02:59:50.102224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 02:59:50.102224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:50.102224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:50.102224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:50.102224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:50.102224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:50.102224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:50.102224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:50.102224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:50.102224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:50.102224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:50.102224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:50.102224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:50.102224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:50.102224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:50.102224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:50.102224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:50.104372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 02:59:50.104372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 02:59:50.104372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 02:59:50.104372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 02:59:50.104372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 02:59:50.104372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 02:59:50.104372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:50.104372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:50.107269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 285/557] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:50.108682 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b3d08-1508-44be-9944-31a4a2b82ec0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:50.194175 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:50.214897 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54498cfc-7bb6-4438-aad6-729513bd6f5b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:50.258523 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b3d08-1508-44be-9944-31a4a2b82ec0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:50.263292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6fb97-769d-4e44-9578-849f21490c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:50.263922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6fb97-769d-4e44-9578-849f21490c31 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:50.264201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6fb97-769d-4e44-9578-849f21490c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:50.265234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6fb97-769d-4e44-9578-849f21490c31 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:50.272713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6fb97-769d-4e44-9578-849f21490c31 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:50.274186 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6fb97-769d-4e44-9578-849f21490c31 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:50.286633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-f6f6fb97-769d-4e44-9578-849f21490c31 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 02:59:50.286633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:50.286633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:50.286633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:50.286633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:50.286633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:50.286633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:50.286633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:50.286633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:50.286633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:50.286633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:50.286633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:50.286633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 02:59:50.286633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 02:59:50.286633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 02:59:50.286633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:50.287693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:50.288656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:50.288656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:50.288656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 02:59:50.288656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 02:59:50.288656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 02:59:50.288656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 02:59:50.288656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 02:59:50.288656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 02:59:50.288656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 02:59:50.288656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:50.288656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 273/558] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 02:59:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:50.288656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6fb97-769d-4e44-9578-849f21490c31 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:50.306513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfe675-b3ae-4721-96c2-95427f2c7afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:50.307556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfe675-b3ae-4721-96c2-95427f2c7afd None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:50.307994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfe675-b3ae-4721-96c2-95427f2c7afd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:50.308396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfe675-b3ae-4721-96c2-95427f2c7afd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:50.324356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfe675-b3ae-4721-96c2-95427f2c7afd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:50.326010 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfe675-b3ae-4721-96c2-95427f2c7afd None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:50.347711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-aabfe675-b3ae-4721-96c2-95427f2c7afd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:50.347711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:50.347711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:50.347711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:50.347711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:50.347711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:50.347711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:50.347711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:50.347711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:50.347711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:50.347711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:50.347711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:50.347711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:50.347711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:50.347711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:50.347711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:50.348927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:50.348927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:50.348927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:50.348927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:50.348927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:50.348927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:50.348927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:50.348927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:50.348927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:50.348927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:50.348927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:50.348927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:50.348927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:50.348927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:50.348927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:50.348927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:50.348927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:50.349819 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:50.349819 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:50.349819 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:50.353776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 286/559] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:50.355405 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad8884-5f13-419a-bf7a-f227b6b294b2 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:50.355699 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad8884-5f13-419a-bf7a-f227b6b294b2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:50.356133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad8884-5f13-419a-bf7a-f227b6b294b2 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:50.361755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 287/560] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:50.363107 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad8884-5f13-419a-bf7a-f227b6b294b2 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:50.412432 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6fb97-769d-4e44-9578-849f21490c31 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:50.421628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:50.447793 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eceb02e-084d-4cce-83e2-f5aed31e483f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:50.448394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eceb02e-084d-4cce-83e2-f5aed31e483f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:50.448671 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eceb02e-084d-4cce-83e2-f5aed31e483f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:50.448941 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eceb02e-084d-4cce-83e2-f5aed31e483f None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:50.449625 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eceb02e-084d-4cce-83e2-f5aed31e483f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:50.449925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eceb02e-084d-4cce-83e2-f5aed31e483f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:50.451401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eceb02e-084d-4cce-83e2-f5aed31e483f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:50.463015 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 274/561] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 02:59:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:50.463802 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eceb02e-084d-4cce-83e2-f5aed31e483f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:50.477214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:50.482644 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad8884-5f13-419a-bf7a-f227b6b294b2 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:50.525896 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc8f83-6e68-4b4c-ad26-f8730ea697a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:50.526853 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc8f83-6e68-4b4c-ad26-f8730ea697a0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:50.527297 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc8f83-6e68-4b4c-ad26-f8730ea697a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:50.527709 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc8f83-6e68-4b4c-ad26-f8730ea697a0 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:50.528563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc8f83-6e68-4b4c-ad26-f8730ea697a0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:50.528883 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc8f83-6e68-4b4c-ad26-f8730ea697a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:50.530894 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc8f83-6e68-4b4c-ad26-f8730ea697a0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:50.553082 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 288/562] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Oct 17 02:59:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:50.557005 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4fbca-9da2-47eb-bb69-33181fc93401 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:50.557366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4fbca-9da2-47eb-bb69-33181fc93401 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:50.557946 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4fbca-9da2-47eb-bb69-33181fc93401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:50.569670 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eceb02e-084d-4cce-83e2-f5aed31e483f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:50.577498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-32b4fbca-9da2-47eb-bb69-33181fc93401 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:50.578694 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-32b4fbca-9da2-47eb-bb69-33181fc93401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:50.607563 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf2849-0b75-4639-8f51-db687aa5c8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:50.608191 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf2849-0b75-4639-8f51-db687aa5c8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:50.608479 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf2849-0b75-4639-8f51-db687aa5c8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:50.608759 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf2849-0b75-4639-8f51-db687aa5c8a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:50.609407 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf2849-0b75-4639-8f51-db687aa5c8a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:50.609716 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf2849-0b75-4639-8f51-db687aa5c8a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:50.611126 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf2849-0b75-4639-8f51-db687aa5c8a4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:50.627650 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 275/563] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:50.627650 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf2849-0b75-4639-8f51-db687aa5c8a4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:50.655253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 289/564] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 02:59:50.656263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4fbca-9da2-47eb-bb69-33181fc93401 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:50.702243 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:50.721167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:50.761418 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf2849-0b75-4639-8f51-db687aa5c8a4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:50.769076 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4fbca-9da2-47eb-bb69-33181fc93401 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:50.809265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947caa33-b573-4a85-8e0e-130e03129299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:50.809613 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b8666-4a50-4651-ab79-3b3b549fa5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:50.809902 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947caa33-b573-4a85-8e0e-130e03129299 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:50.810185 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947caa33-b573-4a85-8e0e-130e03129299 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:50.810461 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947caa33-b573-4a85-8e0e-130e03129299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:50.811276 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b8666-4a50-4651-ab79-3b3b549fa5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:50.811781 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b8666-4a50-4651-ab79-3b3b549fa5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:50.812408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b8666-4a50-4651-ab79-3b3b549fa5e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:50.813668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b8666-4a50-4651-ab79-3b3b549fa5e8 None admin] RBAC: Adding query filter params (role.id=ea0fd0c10f2f4e529270edbe3525d4c6, scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2, user.id=01637fc07ba846aebcb42325bed46216) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:50.814181 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b8666-4a50-4651-ab79-3b3b549fa5e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:50.815849 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b8666-4a50-4651-ab79-3b3b549fa5e8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:50.821742 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947caa33-b573-4a85-8e0e-130e03129299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:50.823203 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947caa33-b573-4a85-8e0e-130e03129299 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:50.823960 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 276/565] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 02:59:50 2022] GET /identity/v3/role_assignments?user.id=01637fc07ba846aebcb42325bed46216&scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2&role.id=ea0fd0c10f2f4e529270edbe3525d4c6 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:50.824951 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b8666-4a50-4651-ab79-3b3b549fa5e8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:50.856947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-947caa33-b573-4a85-8e0e-130e03129299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:50.856947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:50.856947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:50.856947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:50.856947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:50.856947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:50.856947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:50.856947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:50.856947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:50.856947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:50.856947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:50.856947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:50.856947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:50.856947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:50.856947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:50.856947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:50.858053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:50.858053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:50.858053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:50.858053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:50.858053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:50.858053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:50.858053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:50.858053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:50.858053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:50.858053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:50.858053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:50.858053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:50.858053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:50.858053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:50.858053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:50.858053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:50.858053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:50.859006 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:50.859006 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:50.859006 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:50.867950 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 290/566] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:50.869839 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947caa33-b573-4a85-8e0e-130e03129299 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:50.980718 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b8666-4a50-4651-ab79-3b3b549fa5e8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:51.021902 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:51.032712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a766dba-f41b-43a7-970a-10aaa1fd2380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:51.033374 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a766dba-f41b-43a7-970a-10aaa1fd2380 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:51.033662 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a766dba-f41b-43a7-970a-10aaa1fd2380 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:51.033938 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a766dba-f41b-43a7-970a-10aaa1fd2380 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:51.034252 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947caa33-b573-4a85-8e0e-130e03129299 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:51.037170 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:51.047634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a766dba-f41b-43a7-970a-10aaa1fd2380 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:51.050046 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a766dba-f41b-43a7-970a-10aaa1fd2380 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:51.064059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2a766dba-f41b-43a7-970a-10aaa1fd2380 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 02:59:51.064059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:51.064059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:51.064059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:51.064059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:51.064059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:51.064059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:51.064059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:51.064059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:51.064059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:51.064059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:51.064059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:51.064059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 02:59:51.064059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 02:59:51.064059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 02:59:51.064059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:51.065052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:51.066005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:51.066005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:51.066005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 02:59:51.066005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 02:59:51.066005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 02:59:51.066005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 02:59:51.066005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 02:59:51.066005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 02:59:51.066005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 02:59:51.066005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:51.066709 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 277/567] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Oct 17 02:59:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:51.068347 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a766dba-f41b-43a7-970a-10aaa1fd2380 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:51.081708 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d00c23-4642-4545-bbab-3d8fb3374a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:51.083173 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d00c23-4642-4545-bbab-3d8fb3374a4e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:51.083173 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d00c23-4642-4545-bbab-3d8fb3374a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:51.083446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d00c23-4642-4545-bbab-3d8fb3374a4e None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:51.084446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d00c23-4642-4545-bbab-3d8fb3374a4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:51.084446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d00c23-4642-4545-bbab-3d8fb3374a4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:51.085825 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d00c23-4642-4545-bbab-3d8fb3374a4e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:51.100505 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 291/568] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:51.101179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d00c23-4642-4545-bbab-3d8fb3374a4e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:51.185886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a766dba-f41b-43a7-970a-10aaa1fd2380 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:51.226699 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f7230-9269-4849-9a41-a7f2f3b508fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:51.227369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f7230-9269-4849-9a41-a7f2f3b508fc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:51.227645 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f7230-9269-4849-9a41-a7f2f3b508fc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:51.227921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f7230-9269-4849-9a41-a7f2f3b508fc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:51.228523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f7230-9269-4849-9a41-a7f2f3b508fc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:51.229898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73f7230-9269-4849-9a41-a7f2f3b508fc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:51.235482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d00c23-4642-4545-bbab-3d8fb3374a4e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:51.239538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-d73f7230-9269-4849-9a41-a7f2f3b508fc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 02:59:51.239538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:51.239538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 02:59:51.239538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 02:59:51.239538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 02:59:51.239538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 02:59:51.239538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 02:59:51.239538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 02:59:51.239538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 02:59:51.239538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:51.239538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 02:59:51.239538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:51.239538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:51.239538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:51.239538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:51.239538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:51.239538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:51.240532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:51.240532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:51.240532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:51.240532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:51.240532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:51.240532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:51.240532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 02:59:51.240532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 02:59:51.240532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 02:59:51.240532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 02:59:51.240532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:51.240532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:51.240532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:51.240532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:51.240532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:51.240532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:51.241387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:51.241387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:51.241387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:51.241387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:51.241387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:51.241387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:51.241387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:51.241387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:51.241387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:51.241387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:51.241387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 02:59:51.241387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 02:59:51.241387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 02:59:51.241387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:51.241387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 278/569] 10.222.0.27 () {56 vars in 1129 bytes} [Mon Oct 17 02:59:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:51.242230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f7230-9269-4849-9a41-a7f2f3b508fc None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:51.282095 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f533f8-5b24-4f9c-940b-d31068c9e016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:51.283073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f533f8-5b24-4f9c-940b-d31068c9e016 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:51.283565 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f533f8-5b24-4f9c-940b-d31068c9e016 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:51.284039 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f533f8-5b24-4f9c-940b-d31068c9e016 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:51.285049 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f533f8-5b24-4f9c-940b-d31068c9e016 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:51.285623 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f533f8-5b24-4f9c-940b-d31068c9e016 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:51.287521 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f533f8-5b24-4f9c-940b-d31068c9e016 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:51.306900 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 292/570] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Oct 17 02:59:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:51.308160 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f533f8-5b24-4f9c-940b-d31068c9e016 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:51.321629 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:51.324283 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:51.363327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f7230-9269-4849-9a41-a7f2f3b508fc None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:51.401129 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b6772-d3de-4591-8eb3-cb5629afeebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:51.402456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b6772-d3de-4591-8eb3-cb5629afeebe None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:51.402974 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b6772-d3de-4591-8eb3-cb5629afeebe None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:51.403452 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b6772-d3de-4591-8eb3-cb5629afeebe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:51.411308 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b6772-d3de-4591-8eb3-cb5629afeebe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:51.413101 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b6772-d3de-4591-8eb3-cb5629afeebe None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:51.424839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-601b6772-d3de-4591-8eb3-cb5629afeebe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:51.424839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:51.424839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:51.424839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:51.424839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:51.424839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:51.424839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:51.424839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:51.424839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:51.424839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:51.424839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:51.424839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:51.424839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 02:59:51.424839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 02:59:51.424839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 02:59:51.426000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 02:59:51.426000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:51.426000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:51.426000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:51.426000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:51.426000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:51.426000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:51.426000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:51.426000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:51.426000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:51.426000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:51.426000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:51.426000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:51.426000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:51.426000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:51.426000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:51.426000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:51.427057 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 02:59:51.427057 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 02:59:51.427057 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 02:59:51.427057 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 02:59:51.427057 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 02:59:51.427057 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 02:59:51.427057 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:51.427057 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:51.429015 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 279/571] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:51.430021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b6772-d3de-4591-8eb3-cb5629afeebe None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:51.450492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f533f8-5b24-4f9c-940b-d31068c9e016 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:51.498033 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387376e6-905c-40b6-8c6f-3ba7829a7806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:51.498791 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387376e6-905c-40b6-8c6f-3ba7829a7806 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:51.499229 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387376e6-905c-40b6-8c6f-3ba7829a7806 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:51.499526 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387376e6-905c-40b6-8c6f-3ba7829a7806 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:51.500170 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387376e6-905c-40b6-8c6f-3ba7829a7806 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:51.500479 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387376e6-905c-40b6-8c6f-3ba7829a7806 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:51.501873 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387376e6-905c-40b6-8c6f-3ba7829a7806 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:51.510183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 293/572] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Oct 17 02:59:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:51.511141 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387376e6-905c-40b6-8c6f-3ba7829a7806 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:51.554015 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b6772-d3de-4591-8eb3-cb5629afeebe None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:51.574805 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:51.595395 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f76af6-0626-47a4-a891-f20e2db05a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:51.596041 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f76af6-0626-47a4-a891-f20e2db05a04 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:51.596375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f76af6-0626-47a4-a891-f20e2db05a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:51.596651 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f76af6-0626-47a4-a891-f20e2db05a04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:51.610037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f76af6-0626-47a4-a891-f20e2db05a04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:51.612309 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f76af6-0626-47a4-a891-f20e2db05a04 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:51.616929 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:51.630341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-14f76af6-0626-47a4-a891-f20e2db05a04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:51.630341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:51.630341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:51.630341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:51.630341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:51.630341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:51.630341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:51.630341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:51.630341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:51.630341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:51.630341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:51.630341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:51.630341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:51.630341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:51.630341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:51.630341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:51.631684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:51.631684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:51.631684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:51.631684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:51.631684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:51.631684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:51.631684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:51.631684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:51.631684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:51.631684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:51.631684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:51.631684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:51.631684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:51.631684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:51.631684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:51.631684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:51.631684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:51.632609 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:51.632609 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:51.632609 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:51.634946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 280/573] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:51.636039 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f76af6-0626-47a4-a891-f20e2db05a04 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:51.654431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387376e6-905c-40b6-8c6f-3ba7829a7806 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:51.692401 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5bad2-d6eb-4467-ae48-be57755553da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:51.693122 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5bad2-d6eb-4467-ae48-be57755553da None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:51.693501 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5bad2-d6eb-4467-ae48-be57755553da None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:51.693845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c5bad2-d6eb-4467-ae48-be57755553da None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:51.694548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c5bad2-d6eb-4467-ae48-be57755553da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:51.694857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c5bad2-d6eb-4467-ae48-be57755553da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:51.696331 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c5bad2-d6eb-4467-ae48-be57755553da None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:51.713075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 294/574] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 02:59:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:51.713075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5bad2-d6eb-4467-ae48-be57755553da None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:51.757702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f76af6-0626-47a4-a891-f20e2db05a04 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:51.796977 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd44116-8b03-4bb1-bbe5-e93f37117093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:51.797766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd44116-8b03-4bb1-bbe5-e93f37117093 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:51.798110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd44116-8b03-4bb1-bbe5-e93f37117093 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:51.798450 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd44116-8b03-4bb1-bbe5-e93f37117093 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:51.799163 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd44116-8b03-4bb1-bbe5-e93f37117093 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:51.799467 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd44116-8b03-4bb1-bbe5-e93f37117093 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:51.801431 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd44116-8b03-4bb1-bbe5-e93f37117093 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:51.812211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 281/575] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Oct 17 02:59:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:51.812211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d6767-3815-4238-a9dc-2e6f306c8ad8 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:51.812211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d6767-3815-4238-a9dc-2e6f306c8ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:51.812211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d6767-3815-4238-a9dc-2e6f306c8ad8 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:51.812211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 282/576] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:51.812211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d6767-3815-4238-a9dc-2e6f306c8ad8 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:51.828388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c5bad2-d6eb-4467-ae48-be57755553da None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:51.850263 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:51.868289 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd628b-fa72-4b0d-a782-35326f775469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:51.868929 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd628b-fa72-4b0d-a782-35326f775469 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:51.869210 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd628b-fa72-4b0d-a782-35326f775469 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:51.869486 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd628b-fa72-4b0d-a782-35326f775469 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:51.870191 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd628b-fa72-4b0d-a782-35326f775469 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:51.870496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd628b-fa72-4b0d-a782-35326f775469 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:51.872069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcd628b-fa72-4b0d-a782-35326f775469 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:51.874241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:51.888880 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 295/577] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:51.889518 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd628b-fa72-4b0d-a782-35326f775469 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:51.924881 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d6767-3815-4238-a9dc-2e6f306c8ad8 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:51.961729 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8e26e-0883-4458-872c-51ffe7393940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:51.962389 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8e26e-0883-4458-872c-51ffe7393940 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:51.962672 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8e26e-0883-4458-872c-51ffe7393940 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:51.962992 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8e26e-0883-4458-872c-51ffe7393940 None admin] PATH_INFO: `/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/eceda8b3a8ba4233851d0138d8f4d06a/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:51.964946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8e26e-0883-4458-872c-51ffe7393940 None admin] RBAC: Authorizing `identity:create_grant(project_id=89b0ee95955f465eb9ee5b8ae337d9c2, user_id=eceda8b3a8ba4233851d0138d8f4d06a, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:51.966335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8e26e-0883-4458-872c-51ffe7393940 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:51.991025 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 283/578] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 02:59:51 2022] PUT /identity/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/eceda8b3a8ba4233851d0138d8f4d06a/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:59:51.993259 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7ba81-23df-4929-a056-67d89442f87f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:51.993606 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7ba81-23df-4929-a056-67d89442f87f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:51.993893 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7ba81-23df-4929-a056-67d89442f87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:51.998534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd628b-fa72-4b0d-a782-35326f775469 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:52.019544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-def7ba81-23df-4929-a056-67d89442f87f None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:52.021881 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-def7ba81-23df-4929-a056-67d89442f87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:52.041378 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdba04a-acb7-4039-a197-851864fcfb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:52.042782 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdba04a-acb7-4039-a197-851864fcfb59 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:52.043355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdba04a-acb7-4039-a197-851864fcfb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:52.043826 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdba04a-acb7-4039-a197-851864fcfb59 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:52.044840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdba04a-acb7-4039-a197-851864fcfb59 None admin] RBAC: Adding query filter params (interface=public, service_id=4b009bc92b624644bdea6b669e8d6cfb, region_id=RegionOne) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:52.045374 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdba04a-acb7-4039-a197-851864fcfb59 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:52.047098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdba04a-acb7-4039-a197-851864fcfb59 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:52.057215 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 296/579] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Oct 17 02:59:51 2022] GET /identity/v3/endpoints?service_id=4b009bc92b624644bdea6b669e8d6cfb&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:52.063184 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdba04a-acb7-4039-a197-851864fcfb59 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:52.111357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 284/580] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 02:59:52.112903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7ba81-23df-4929-a056-67d89442f87f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:52.141254 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:52.149613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:52.208987 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdba04a-acb7-4039-a197-851864fcfb59 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:52.230168 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7ba81-23df-4929-a056-67d89442f87f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:52.252169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a5726-de87-4871-ad66-21591469c416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:52.252816 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a5726-de87-4871-ad66-21591469c416 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:52.253099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a5726-de87-4871-ad66-21591469c416 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:52.253377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a5726-de87-4871-ad66-21591469c416 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:52.262444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a5726-de87-4871-ad66-21591469c416 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:52.264365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a5726-de87-4871-ad66-21591469c416 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:52.274113 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11559652-860f-4d97-b223-c5618ef753fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:52.275049 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11559652-860f-4d97-b223-c5618ef753fe None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:52.275565 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11559652-860f-4d97-b223-c5618ef753fe None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:52.276040 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11559652-860f-4d97-b223-c5618ef753fe None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:52.276933 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11559652-860f-4d97-b223-c5618ef753fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:52.277726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d36a5726-de87-4871-ad66-21591469c416 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:52.277726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:52.277726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:52.277726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:52.277726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:52.277726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:52.277726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:52.277726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:52.277726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:52.277726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:52.277726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:52.277726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:52.277726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 02:59:52.277726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 02:59:52.277726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 02:59:52.278592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 02:59:52.278592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:52.278592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:52.278592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:52.278592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:52.278592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:52.278592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:52.278592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:52.278592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:52.278592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:52.278592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:52.278592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:52.278592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:52.278592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:52.278592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:52.278592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:52.278592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:52.279756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 02:59:52.279756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 02:59:52.279756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 02:59:52.279756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 02:59:52.279756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 02:59:52.279756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 02:59:52.279756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:52.279756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:52.282085 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11559652-860f-4d97-b223-c5618ef753fe None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:52.283229 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 297/581] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:52.286068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a5726-de87-4871-ad66-21591469c416 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:52.289534 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 285/582] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Oct 17 02:59:52 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:52.399566 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a5726-de87-4871-ad66-21591469c416 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:52.433785 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:52.441228 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e1ef8-00e3-4462-85e8-c82371cbbec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:52.442213 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e1ef8-00e3-4462-85e8-c82371cbbec7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:52.442444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e1ef8-00e3-4462-85e8-c82371cbbec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:52.442724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e1ef8-00e3-4462-85e8-c82371cbbec7 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:52.447751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e1ef8-00e3-4462-85e8-c82371cbbec7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:52.448070 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e1ef8-00e3-4462-85e8-c82371cbbec7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:52.449569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e1ef8-00e3-4462-85e8-c82371cbbec7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:52.463905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 298/583] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 02:59:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:52.469683 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11559652-860f-4d97-b223-c5618ef753fe None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:52.514335 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e1ef8-00e3-4462-85e8-c82371cbbec7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:52.589466 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:52.602811 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11559652-860f-4d97-b223-c5618ef753fe None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:52.643205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62832897-db6b-41f9-b201-57971d4c046c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:52.643978 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62832897-db6b-41f9-b201-57971d4c046c None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:52.644260 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62832897-db6b-41f9-b201-57971d4c046c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:52.644538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62832897-db6b-41f9-b201-57971d4c046c None admin] PATH_INFO: `/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/280d47ed79f14105a83b18b2cac58b60/roles/78dc08989b32455389b21096f5d57e17` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:52.651039 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e1ef8-00e3-4462-85e8-c82371cbbec7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:52.661975 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62832897-db6b-41f9-b201-57971d4c046c None admin] RBAC: Authorizing `identity:create_grant(project_id=89b0ee95955f465eb9ee5b8ae337d9c2, user_id=280d47ed79f14105a83b18b2cac58b60, role_id=78dc08989b32455389b21096f5d57e17)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:52.663572 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62832897-db6b-41f9-b201-57971d4c046c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:52.679317 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 286/584] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 02:59:52 2022] PUT /identity/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/280d47ed79f14105a83b18b2cac58b60/roles/78dc08989b32455389b21096f5d57e17 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 02:59:52.701898 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d33ed-8073-44ca-af54-f80c4e3b0ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:52.702574 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d33ed-8073-44ca-af54-f80c4e3b0ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:52.702859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d33ed-8073-44ca-af54-f80c4e3b0ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:52.703184 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d33ed-8073-44ca-af54-f80c4e3b0ca7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:52.703784 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d33ed-8073-44ca-af54-f80c4e3b0ca7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:52.705207 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16d33ed-8073-44ca-af54-f80c4e3b0ca7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:52.715561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e16d33ed-8073-44ca-af54-f80c4e3b0ca7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 02:59:52.715561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:52.715561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 02:59:52.715561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 02:59:52.715561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 02:59:52.715561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 02:59:52.715561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 02:59:52.715561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 02:59:52.715561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 02:59:52.715561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:52.715561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 02:59:52.715561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:52.715561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:52.715561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:52.715561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:52.715561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:52.715561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:52.716755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:52.716755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:52.716755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:52.716755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:52.716755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:52.716755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:52.716755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 02:59:52.716755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 02:59:52.716755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 02:59:52.716755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 02:59:52.716755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:52.716755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:52.716755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:52.716755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:52.716755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:52.716755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:52.717697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:52.717697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:52.717697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:52.717697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:52.717697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:52.717697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:52.717697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:52.717697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:52.717697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:52.717697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:52.717697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 02:59:52.717697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 02:59:52.717697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 02:59:52.717697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:52.717697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 299/585] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:52 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:59:52.728290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62832897-db6b-41f9-b201-57971d4c046c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:52.889770 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62832897-db6b-41f9-b201-57971d4c046c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:52.924877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101973a-bd1a-441b-a6b7-7732e6a1b477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:52.925722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101973a-bd1a-441b-a6b7-7732e6a1b477 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:52.926276 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101973a-bd1a-441b-a6b7-7732e6a1b477 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:52.926699 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101973a-bd1a-441b-a6b7-7732e6a1b477 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:52.927511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101973a-bd1a-441b-a6b7-7732e6a1b477 None admin] RBAC: Adding query filter params (name=network) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:52.927969 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101973a-bd1a-441b-a6b7-7732e6a1b477 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:52.929484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101973a-bd1a-441b-a6b7-7732e6a1b477 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:52.934347 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 287/586] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Oct 17 02:59:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:52.940058 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d33ed-8073-44ca-af54-f80c4e3b0ca7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:52.988227 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:53.077635 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d33ed-8073-44ca-af54-f80c4e3b0ca7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:53.128677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d84e00-1021-4651-a2f9-ca97a1896c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:53.130508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d84e00-1021-4651-a2f9-ca97a1896c63 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:53.131287 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d84e00-1021-4651-a2f9-ca97a1896c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:53.131775 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d84e00-1021-4651-a2f9-ca97a1896c63 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:53.132627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d84e00-1021-4651-a2f9-ca97a1896c63 None admin] RBAC: Adding query filter params (type=network) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:53.133141 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d84e00-1021-4651-a2f9-ca97a1896c63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:53.134739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d84e00-1021-4651-a2f9-ca97a1896c63 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:53.142814 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 300/587] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Oct 17 02:59:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:53.148725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101973a-bd1a-441b-a6b7-7732e6a1b477 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:53.157504 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:53.315004 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101973a-bd1a-441b-a6b7-7732e6a1b477 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:53.352776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059d1a1-d9af-4042-b9a6-5acf0db89581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:53.353384 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059d1a1-d9af-4042-b9a6-5acf0db89581 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:53.353665 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059d1a1-d9af-4042-b9a6-5acf0db89581 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:53.353941 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059d1a1-d9af-4042-b9a6-5acf0db89581 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:53.354647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059d1a1-d9af-4042-b9a6-5acf0db89581 None admin] RBAC: Adding query filter params (interface=public, service_id=c8549009f4414a72a252609dd4e81976, region_id=RegionOne) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:53.354983 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059d1a1-d9af-4042-b9a6-5acf0db89581 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:53.356353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059d1a1-d9af-4042-b9a6-5acf0db89581 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:53.364708 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 288/588] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Oct 17 02:59:53 2022] GET /identity/v3/endpoints?service_id=c8549009f4414a72a252609dd4e81976&interface=public®ion_id=RegionOne => generated 194 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:53.369695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d84e00-1021-4651-a2f9-ca97a1896c63 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:53.395851 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2eb74-6ed3-4913-803b-bdcca922fa44 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:53.396039 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2eb74-6ed3-4913-803b-bdcca922fa44 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:53.396322 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2eb74-6ed3-4913-803b-bdcca922fa44 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:53.397703 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 289/589] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 02:59:53.405062 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009ba6f-ddd2-45b2-af0c-f3380addbe2d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:53.405062 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009ba6f-ddd2-45b2-af0c-f3380addbe2d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:53.405062 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009ba6f-ddd2-45b2-af0c-f3380addbe2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:53.425623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c009ba6f-ddd2-45b2-af0c-f3380addbe2d None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:53.427760 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c009ba6f-ddd2-45b2-af0c-f3380addbe2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:53.517380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:53.532668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 290/590] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 02:59:53.568356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d84e00-1021-4651-a2f9-ca97a1896c63 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:53.577495 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:53.617978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8685c2-adad-4103-901f-348519f4e401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:53.618651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8685c2-adad-4103-901f-348519f4e401 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:53.618964 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8685c2-adad-4103-901f-348519f4e401 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:53.619244 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8685c2-adad-4103-901f-348519f4e401 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:53.619863 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8685c2-adad-4103-901f-348519f4e401 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:53.621230 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8685c2-adad-4103-901f-348519f4e401 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:53.627739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 301/591] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Oct 17 02:59:53 2022] GET /identity/v3/services => generated 772 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:53.922669 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009ba6f-ddd2-45b2-af0c-f3380addbe2d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:54.030069 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009ba6f-ddd2-45b2-af0c-f3380addbe2d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:54.064550 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871db9f-4598-43ad-9747-dddcf16d2268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:54.065186 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871db9f-4598-43ad-9747-dddcf16d2268 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:54.065468 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871db9f-4598-43ad-9747-dddcf16d2268 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:54.065741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871db9f-4598-43ad-9747-dddcf16d2268 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:54.066328 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871db9f-4598-43ad-9747-dddcf16d2268 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:54.067810 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871db9f-4598-43ad-9747-dddcf16d2268 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:54.076915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-d871db9f-4598-43ad-9747-dddcf16d2268 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 02:59:54.076915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:54.076915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 02:59:54.076915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 02:59:54.076915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 02:59:54.076915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 02:59:54.076915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 02:59:54.076915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 02:59:54.076915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 02:59:54.076915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:54.076915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 02:59:54.076915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:54.076915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:54.076915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:54.076915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:54.076915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:54.077876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:54.077876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:54.077876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:54.077876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:54.077876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:54.077876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:54.077876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:54.077876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 02:59:54.077876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 02:59:54.077876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 02:59:54.077876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 02:59:54.077876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:54.077876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:54.077876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:54.077876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:54.077876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:54.077876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:54.078768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:54.078768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:54.078768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:54.078768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:54.078768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:54.078768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:54.078768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:54.078768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:54.078768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:54.078768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:54.078768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 02:59:54.078768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 02:59:54.078768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 02:59:54.078768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:54.078768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 291/592] 10.222.0.27 () {56 vars in 1132 bytes} [Mon Oct 17 02:59:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:54.087042 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8685c2-adad-4103-901f-348519f4e401 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:54.227449 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8685c2-adad-4103-901f-348519f4e401 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:54.284512 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c854a16-0579-4740-90d0-a0498c47e07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:54.285142 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c854a16-0579-4740-90d0-a0498c47e07e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:54.285487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c854a16-0579-4740-90d0-a0498c47e07e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:54.285765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c854a16-0579-4740-90d0-a0498c47e07e None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:54.286401 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c854a16-0579-4740-90d0-a0498c47e07e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:54.286708 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c854a16-0579-4740-90d0-a0498c47e07e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:54.288111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c854a16-0579-4740-90d0-a0498c47e07e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:54.297156 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 302/593] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Oct 17 02:59:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:54.302942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871db9f-4598-43ad-9747-dddcf16d2268 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:54.389278 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:54.447406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871db9f-4598-43ad-9747-dddcf16d2268 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:54.491336 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55959e-b155-43fd-b8b2-fbb81c8d4e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:54.491893 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55959e-b155-43fd-b8b2-fbb81c8d4e43 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:54.492175 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55959e-b155-43fd-b8b2-fbb81c8d4e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:54.492456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55959e-b155-43fd-b8b2-fbb81c8d4e43 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:54.493114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55959e-b155-43fd-b8b2-fbb81c8d4e43 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:54.493419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55959e-b155-43fd-b8b2-fbb81c8d4e43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:54.494894 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55959e-b155-43fd-b8b2-fbb81c8d4e43 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:54.500645 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 292/594] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Oct 17 02:59:54 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:54.597000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8573e-a99d-40d3-a569-4cb5042f2b98 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:54.597385 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8573e-a99d-40d3-a569-4cb5042f2b98 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:54.597550 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8573e-a99d-40d3-a569-4cb5042f2b98 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:54.599791 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 303/595] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:54.605689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcba708-8367-4b52-96d7-f602e86583c8 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:54.605964 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcba708-8367-4b52-96d7-f602e86583c8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:54.606245 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcba708-8367-4b52-96d7-f602e86583c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:54.625708 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-afcba708-8367-4b52-96d7-f602e86583c8 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:54.627011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-afcba708-8367-4b52-96d7-f602e86583c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:54.729198 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 293/596] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 02:59:55.081633 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8573e-a99d-40d3-a569-4cb5042f2b98 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:55.095257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f950af5-0138-4e53-aec0-801722078cdf None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:55.095885 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f950af5-0138-4e53-aec0-801722078cdf None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:55.096365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f950af5-0138-4e53-aec0-801722078cdf None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:55.099747 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 294/597] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:55.104869 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3514e-fd18-4989-9eb3-0fb5cd3ff148 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:55.105415 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3514e-fd18-4989-9eb3-0fb5cd3ff148 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:55.105882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3514e-fd18-4989-9eb3-0fb5cd3ff148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:55.141419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-00d3514e-fd18-4989-9eb3-0fb5cd3ff148 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:55.144877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-00d3514e-fd18-4989-9eb3-0fb5cd3ff148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:55.148358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:55.247279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8573e-a99d-40d3-a569-4cb5042f2b98 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:55.257120 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:55.286684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347fbdd-a5b7-416a-95bc-12fb07ecd492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:55.287668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347fbdd-a5b7-416a-95bc-12fb07ecd492 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:55.288167 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347fbdd-a5b7-416a-95bc-12fb07ecd492 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:55.288670 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347fbdd-a5b7-416a-95bc-12fb07ecd492 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:55.298803 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 295/598] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 02:59:55.300747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347fbdd-a5b7-416a-95bc-12fb07ecd492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:55.306411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-6347fbdd-a5b7-416a-95bc-12fb07ecd492 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:59:55.306411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:55.306411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:55.306411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:55.306411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:55.306411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:55.306411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:55.306411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:55.306411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:55.306411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:55.306411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:55.306411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:55.306411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:55.306411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:55.306411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:55.306411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:55.307390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:55.307390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:55.307390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:55.307390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:55.307390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:55.307390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:55.307390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:55.307390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:55.307390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:55.307390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:55.307390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:55.307390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:55.307390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:55.307390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:55.307390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:55.307390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:55.308233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:55.308233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:55.308233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:55.308233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:55.308233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:55.308233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:55.308233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:55.308233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:55.308233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:55.308233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:55.308233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 02:59:55.308233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:55.310812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 304/599] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 02:59:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:55.318916 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3514e-fd18-4989-9eb3-0fb5cd3ff148 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:55.467336 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3514e-fd18-4989-9eb3-0fb5cd3ff148 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:55.484394 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c42a5-b61d-4b78-862c-033369bb9222 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:55.484719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c42a5-b61d-4b78-862c-033369bb9222 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:55.484951 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c42a5-b61d-4b78-862c-033369bb9222 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:55.486274 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 305/600] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:55.498356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fc19c-b9b5-4a83-bb54-3479de1d3ed4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:55.498671 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fc19c-b9b5-4a83-bb54-3479de1d3ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:55.498949 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fc19c-b9b5-4a83-bb54-3479de1d3ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:55.520524 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc35bdf-4070-43eb-8095-d4f7793d01ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:55.521202 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc35bdf-4070-43eb-8095-d4f7793d01ae None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:55.521484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc35bdf-4070-43eb-8095-d4f7793d01ae None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:55.521754 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc35bdf-4070-43eb-8095-d4f7793d01ae None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:55.522441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc35bdf-4070-43eb-8095-d4f7793d01ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:55.522750 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc35bdf-4070-43eb-8095-d4f7793d01ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:55.525558 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc35bdf-4070-43eb-8095-d4f7793d01ae None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:55.535800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-661fc19c-b9b5-4a83-bb54-3479de1d3ed4 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:55.537009 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-661fc19c-b9b5-4a83-bb54-3479de1d3ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:55.543027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 296/601] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 02:59:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:55.548327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc35bdf-4070-43eb-8095-d4f7793d01ae None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:55.620161 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:55.651077 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 306/602] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 02:59:55.652313 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fc19c-b9b5-4a83-bb54-3479de1d3ed4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:55.678264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:55.692715 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc35bdf-4070-43eb-8095-d4f7793d01ae None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:55.740450 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87594ef3-fc7a-4a99-8cec-ac025b223717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:55.741016 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87594ef3-fc7a-4a99-8cec-ac025b223717 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:55.741302 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87594ef3-fc7a-4a99-8cec-ac025b223717 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:55.741576 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87594ef3-fc7a-4a99-8cec-ac025b223717 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:55.757097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87594ef3-fc7a-4a99-8cec-ac025b223717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:55.759337 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87594ef3-fc7a-4a99-8cec-ac025b223717 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:55.760878 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fc19c-b9b5-4a83-bb54-3479de1d3ed4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:55.779239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-87594ef3-fc7a-4a99-8cec-ac025b223717 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:55.779239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:55.779239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:55.779239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:55.779239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:55.779239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:55.779239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:55.779239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:55.779239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:55.779239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:55.779239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:55.779239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:55.779239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:55.779239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:55.779239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:55.779239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:55.780255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:55.780255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:55.780255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:55.780255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:55.780255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:55.780255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:55.780255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:55.780255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:55.780255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:55.780255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:55.780255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:55.780255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:55.780255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:55.780255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:55.780255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:55.780255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:55.780255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:55.781155 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:55.781155 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:55.781155 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:55.781155 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 297/603] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:59:55.787743 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87594ef3-fc7a-4a99-8cec-ac025b223717 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:55.834523 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ea168-734b-4454-86f8-a92ef7716d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:55.835540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ea168-734b-4454-86f8-a92ef7716d61 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:55.836017 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ea168-734b-4454-86f8-a92ef7716d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:55.836477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ea168-734b-4454-86f8-a92ef7716d61 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:55.837244 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ea168-734b-4454-86f8-a92ef7716d61 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:55.838938 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ea168-734b-4454-86f8-a92ef7716d61 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:55.856975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e90ea168-734b-4454-86f8-a92ef7716d61 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 02:59:55.856975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:55.856975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 02:59:55.856975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 02:59:55.856975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 02:59:55.856975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 02:59:55.856975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 02:59:55.856975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 02:59:55.856975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 02:59:55.856975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:55.856975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 02:59:55.856975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:55.856975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:55.856975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:55.856975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:55.856975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:55.856975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:55.858007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:55.858007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:55.858007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:55.858007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:55.858007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:55.858007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:55.858007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 02:59:55.858007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 02:59:55.858007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 02:59:55.858007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 02:59:55.858007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:55.858007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:55.858007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:55.858007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:55.858007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:55.858007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:55.858897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:55.858897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:55.858897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:55.858897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:55.858897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:55.858897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:55.858897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:55.858897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:55.858897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:55.858897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:55.858897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 02:59:55.858897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 02:59:55.858897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 02:59:55.858897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:55.867316 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 307/604] 10.222.0.27 () {56 vars in 1129 bytes} [Mon Oct 17 02:59:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:55.873141 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ea168-734b-4454-86f8-a92ef7716d61 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:55.907253 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:55.914889 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87594ef3-fc7a-4a99-8cec-ac025b223717 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:55.954447 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea006ef-e8c5-476d-b608-cc62f782746f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:55.955271 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea006ef-e8c5-476d-b608-cc62f782746f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:55.955933 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea006ef-e8c5-476d-b608-cc62f782746f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:55.956326 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea006ef-e8c5-476d-b608-cc62f782746f None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:55.957112 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea006ef-e8c5-476d-b608-cc62f782746f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:55.957575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea006ef-e8c5-476d-b608-cc62f782746f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:55.959396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea006ef-e8c5-476d-b608-cc62f782746f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:55.974769 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 298/605] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:55.980306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea006ef-e8c5-476d-b608-cc62f782746f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:56.023664 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:56.039014 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ea168-734b-4454-86f8-a92ef7716d61 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:56.083116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806eeb5-e247-4509-91ef-4fc623baba44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:56.083899 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806eeb5-e247-4509-91ef-4fc623baba44 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:56.084356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806eeb5-e247-4509-91ef-4fc623baba44 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:56.084737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806eeb5-e247-4509-91ef-4fc623baba44 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:56.086028 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806eeb5-e247-4509-91ef-4fc623baba44 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:56.086443 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806eeb5-e247-4509-91ef-4fc623baba44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:56.088746 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806eeb5-e247-4509-91ef-4fc623baba44 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:56.095962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 308/606] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Oct 17 02:59:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:56.097144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806eeb5-e247-4509-91ef-4fc623baba44 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:56.123867 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea006ef-e8c5-476d-b608-cc62f782746f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:56.172047 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a713cc-9147-420f-80ae-5edc07e5ae4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:56.173717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a713cc-9147-420f-80ae-5edc07e5ae4d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:56.174188 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a713cc-9147-420f-80ae-5edc07e5ae4d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:56.174623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a713cc-9147-420f-80ae-5edc07e5ae4d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:56.184143 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a713cc-9147-420f-80ae-5edc07e5ae4d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:56.186094 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a713cc-9147-420f-80ae-5edc07e5ae4d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:56.201823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-c7a713cc-9147-420f-80ae-5edc07e5ae4d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 02:59:56.201823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:56.201823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:56.201823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:56.201823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:56.201823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:56.201823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:56.201823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:56.201823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:56.201823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:56.201823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:56.201823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:56.201823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 02:59:56.201823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 02:59:56.201823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 02:59:56.201823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:56.202836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:56.203836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:56.203836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:56.203836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 02:59:56.203836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 02:59:56.203836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 02:59:56.203836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 02:59:56.203836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 02:59:56.203836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 02:59:56.203836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 02:59:56.203836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:56.204994 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 299/607] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 02:59:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:56.206112 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a713cc-9147-420f-80ae-5edc07e5ae4d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:56.210170 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806eeb5-e247-4509-91ef-4fc623baba44 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:56.234096 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:56.258518 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80309c6-f00f-4268-acab-5915f3bda04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:56.260167 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80309c6-f00f-4268-acab-5915f3bda04a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:56.260167 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80309c6-f00f-4268-acab-5915f3bda04a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:56.260167 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80309c6-f00f-4268-acab-5915f3bda04a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:56.276139 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80309c6-f00f-4268-acab-5915f3bda04a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 02:59:56.283933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d80309c6-f00f-4268-acab-5915f3bda04a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:56.283933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:56.283933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:56.283933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:56.283933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:56.283933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:56.283933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:56.283933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:56.283933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:56.283933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:56.283933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:56.283933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:56.283933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 02:59:56.283933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 02:59:56.283933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 02:59:56.283933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 02:59:56.284996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 02:59:56.284996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 02:59:56.284996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 02:59:56.284996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 02:59:56.284996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:56.284996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:56.284996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:56.284996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:56.284996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:56.284996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:56.284996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:56.284996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:56.284996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:56.284996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:56.284996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:56.284996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:56.286161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:56.286161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:56.286161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:56.286161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:56.286161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 02:59:56.286161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 02:59:56.286161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 02:59:56.286161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 02:59:56.286161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 02:59:56.286161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 02:59:56.286161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 02:59:56.286161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:56.286161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 309/608] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Oct 17 02:59:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:56.287001 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80309c6-f00f-4268-acab-5915f3bda04a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:56.310250 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:56.348467 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a713cc-9147-420f-80ae-5edc07e5ae4d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:56.384882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd23c72-446a-4d45-98e6-b5075d657587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:56.385869 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd23c72-446a-4d45-98e6-b5075d657587 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:56.386347 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd23c72-446a-4d45-98e6-b5075d657587 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:56.386923 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd23c72-446a-4d45-98e6-b5075d657587 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:56.387856 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd23c72-446a-4d45-98e6-b5075d657587 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:56.388381 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd23c72-446a-4d45-98e6-b5075d657587 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:56.389992 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd23c72-446a-4d45-98e6-b5075d657587 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:56.400878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 300/609] 10.222.0.27 () {56 vars in 1124 bytes} [Mon Oct 17 02:59:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:56.402196 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd23c72-446a-4d45-98e6-b5075d657587 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:56.431576 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80309c6-f00f-4268-acab-5915f3bda04a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:56.466542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6d2d4-bf50-4897-a815-4ce6f370bafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:56.467171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6d2d4-bf50-4897-a815-4ce6f370bafb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:56.467482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6d2d4-bf50-4897-a815-4ce6f370bafb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:56.467714 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6d2d4-bf50-4897-a815-4ce6f370bafb None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:56.468432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6d2d4-bf50-4897-a815-4ce6f370bafb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:56.468739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6d2d4-bf50-4897-a815-4ce6f370bafb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:56.470144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6d2d4-bf50-4897-a815-4ce6f370bafb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:56.487886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 310/610] 10.222.0.27 () {56 vars in 1135 bytes} [Mon Oct 17 02:59:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:56.489227 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6d2d4-bf50-4897-a815-4ce6f370bafb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:56.494896 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:56.515943 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd23c72-446a-4d45-98e6-b5075d657587 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:56.552434 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588231a-ed06-47e6-aba8-47de538cd02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:56.553046 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588231a-ed06-47e6-aba8-47de538cd02e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:56.553324 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588231a-ed06-47e6-aba8-47de538cd02e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:56.553596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588231a-ed06-47e6-aba8-47de538cd02e None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:56.554257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588231a-ed06-47e6-aba8-47de538cd02e None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:56.554560 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588231a-ed06-47e6-aba8-47de538cd02e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:56.556193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588231a-ed06-47e6-aba8-47de538cd02e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:56.559297 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:56.570489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 301/611] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Oct 17 02:59:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:56.570489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d8848-32ec-42a3-b2e8-a358eb1d7e53 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:56.570489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d8848-32ec-42a3-b2e8-a358eb1d7e53 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:56.570489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d8848-32ec-42a3-b2e8-a358eb1d7e53 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:56.570489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 302/612] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:56.574088 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d8848-32ec-42a3-b2e8-a358eb1d7e53 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:56.596313 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6d2d4-bf50-4897-a815-4ce6f370bafb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:56.633852 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a100f1-25ae-4401-a2c4-114ea9b61dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:56.634476 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a100f1-25ae-4401-a2c4-114ea9b61dfb None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:56.634762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a100f1-25ae-4401-a2c4-114ea9b61dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:56.635067 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a100f1-25ae-4401-a2c4-114ea9b61dfb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:56.635734 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a100f1-25ae-4401-a2c4-114ea9b61dfb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:56.637093 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a100f1-25ae-4401-a2c4-114ea9b61dfb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:56.662310 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 311/613] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 02:59:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:59:56.663438 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a100f1-25ae-4401-a2c4-114ea9b61dfb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:56.702775 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d8848-32ec-42a3-b2e8-a358eb1d7e53 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:56.747417 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367de5ae-42dc-4793-983f-fd64adef5f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:56.748365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367de5ae-42dc-4793-983f-fd64adef5f55 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:56.749124 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367de5ae-42dc-4793-983f-fd64adef5f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:56.749561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367de5ae-42dc-4793-983f-fd64adef5f55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:56.766397 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:56.788408 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a100f1-25ae-4401-a2c4-114ea9b61dfb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:56.819157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:56.831966 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e6eca6-1d01-4b06-b3b1-51a441b99cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:56.832507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6eca6-1d01-4b06-b3b1-51a441b99cfa None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:56.833327 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6eca6-1d01-4b06-b3b1-51a441b99cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:56.834119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6eca6-1d01-4b06-b3b1-51a441b99cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:56.848795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6eca6-1d01-4b06-b3b1-51a441b99cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:56.850444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e6eca6-1d01-4b06-b3b1-51a441b99cfa None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:56.856528 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367de5ae-42dc-4793-983f-fd64adef5f55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:56.859084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367de5ae-42dc-4793-983f-fd64adef5f55 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:56.874591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-72e6eca6-1d01-4b06-b3b1-51a441b99cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:56.874591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:56.874591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:56.874591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:56.874591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:56.874591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:56.874591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:56.874591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:56.874591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:56.874591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:56.874591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:56.874591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:56.874591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:56.874591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:56.874591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:56.874591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:56.875847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:56.875847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:56.875847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:56.875847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:56.875847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:56.875847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:56.875847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:56.875847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:56.875847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:56.875847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:56.875847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:56.875847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:56.875847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:56.875847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:56.875847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:56.875847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:56.875847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:56.877215 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:56.877215 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:56.877215 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:56.879967 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 312/614] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:56.882669 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd02fd-7d00-4ef6-8e09-7b2620d2598d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:56.883318 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd02fd-7d00-4ef6-8e09-7b2620d2598d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:56.883857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd02fd-7d00-4ef6-8e09-7b2620d2598d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:56.910123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e3cd02fd-7d00-4ef6-8e09-7b2620d2598d None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:56.910123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e3cd02fd-7d00-4ef6-8e09-7b2620d2598d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:56.985797 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 303/615] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:59:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 02:59:56.986898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367de5ae-42dc-4793-983f-fd64adef5f55 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:57.003675 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 313/616] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 02:59:57.007387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd02fd-7d00-4ef6-8e09-7b2620d2598d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:57.053969 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:57.156088 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367de5ae-42dc-4793-983f-fd64adef5f55 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:57.174074 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd02fd-7d00-4ef6-8e09-7b2620d2598d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:57.199213 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:57.212535 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27dd0d-71a0-4c38-986c-bb2af85f4297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:57.216446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27dd0d-71a0-4c38-986c-bb2af85f4297 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:57.216850 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27dd0d-71a0-4c38-986c-bb2af85f4297 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:57.217224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27dd0d-71a0-4c38-986c-bb2af85f4297 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:57.239992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27dd0d-71a0-4c38-986c-bb2af85f4297 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:57.241710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27dd0d-71a0-4c38-986c-bb2af85f4297 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:57.257333 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539af8c-c905-4167-aa85-a09d095d392e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:57.257333 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539af8c-c905-4167-aa85-a09d095d392e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:57.257333 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539af8c-c905-4167-aa85-a09d095d392e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:57.257333 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539af8c-c905-4167-aa85-a09d095d392e None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:57.257725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539af8c-c905-4167-aa85-a09d095d392e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:57.257725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539af8c-c905-4167-aa85-a09d095d392e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:57.257725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539af8c-c905-4167-aa85-a09d095d392e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:57.268202 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 304/617] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:57.270696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-cd27dd0d-71a0-4c38-986c-bb2af85f4297 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:57.270696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:57.270696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:57.270696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:57.270696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:57.270696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:57.270696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:57.270696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:57.270696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:57.270696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:57.270696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:57.270696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:57.270696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:57.270696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:57.270696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:57.270696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:57.271706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:57.271706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:57.271706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:57.271706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:57.271706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:57.271706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:57.271706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:57.271706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:57.271706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:57.271706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:57.271706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:57.271706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:57.271706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:57.271706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:57.271706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:57.271706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:57.271706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:57.272662 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:57.272662 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:57.272662 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:57.274191 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 314/618] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:57.282428 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539af8c-c905-4167-aa85-a09d095d392e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:57.282986 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27dd0d-71a0-4c38-986c-bb2af85f4297 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:57.441355 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:57.445989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539af8c-c905-4167-aa85-a09d095d392e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:57.449998 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27dd0d-71a0-4c38-986c-bb2af85f4297 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:57.481868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbfb90-6557-43c0-81f3-9f30a11b3bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:57.482595 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbfb90-6557-43c0-81f3-9f30a11b3bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:57.482915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbfb90-6557-43c0-81f3-9f30a11b3bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:57.483194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbfb90-6557-43c0-81f3-9f30a11b3bb8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:57.486181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a528ce3-bd4c-412a-9745-03380e09fba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:57.486804 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a528ce3-bd4c-412a-9745-03380e09fba8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:57.487118 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a528ce3-bd4c-412a-9745-03380e09fba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:57.487394 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a528ce3-bd4c-412a-9745-03380e09fba8 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:57.488038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a528ce3-bd4c-412a-9745-03380e09fba8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:57.488453 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a528ce3-bd4c-412a-9745-03380e09fba8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:57.490144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a528ce3-bd4c-412a-9745-03380e09fba8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:57.491836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbfb90-6557-43c0-81f3-9f30a11b3bb8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:57.493266 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbfb90-6557-43c0-81f3-9f30a11b3bb8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:57.506905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 315/619] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:57.506905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a528ce3-bd4c-412a-9745-03380e09fba8 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:57.507178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-70fbfb90-6557-43c0-81f3-9f30a11b3bb8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 02:59:57.507178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:57.507178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:57.507178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:57.507178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:57.507178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:57.507178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:57.507178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:57.507178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:57.507178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:57.507178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:57.507178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:57.507178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 02:59:57.507178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 02:59:57.507178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 02:59:57.507178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:57.508133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:57.509084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:57.509084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:57.509084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 02:59:57.509084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 02:59:57.509084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 02:59:57.509084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 02:59:57.509084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 02:59:57.509084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 02:59:57.509084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 02:59:57.509084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:57.512142 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 305/620] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Oct 17 02:59:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:57.513430 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2ab49-db6f-459c-9bea-448c46377c87 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:57.513715 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2ab49-db6f-459c-9bea-448c46377c87 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:57.513988 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2ab49-db6f-459c-9bea-448c46377c87 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:57.515420 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 306/621] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:57.520133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2ab49-db6f-459c-9bea-448c46377c87 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:57.531239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:57.616972 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a528ce3-bd4c-412a-9745-03380e09fba8 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:57.628604 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2ab49-db6f-459c-9bea-448c46377c87 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:57.654081 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8a04e-bc12-4883-81f2-43f39787fec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:57.654853 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8a04e-bc12-4883-81f2-43f39787fec7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:57.655164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8a04e-bc12-4883-81f2-43f39787fec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:57.655462 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8a04e-bc12-4883-81f2-43f39787fec7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:57.656152 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8a04e-bc12-4883-81f2-43f39787fec7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:57.657562 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8a04e-bc12-4883-81f2-43f39787fec7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:57.664900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae7043-6a60-441a-bae5-93d376f8bad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:57.665548 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae7043-6a60-441a-bae5-93d376f8bad9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:57.665830 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae7043-6a60-441a-bae5-93d376f8bad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:57.666105 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae7043-6a60-441a-bae5-93d376f8bad9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:57.668725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-68e8a04e-bc12-4883-81f2-43f39787fec7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 02:59:57.668725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:57.668725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 02:59:57.668725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 02:59:57.668725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 02:59:57.668725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 02:59:57.668725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 02:59:57.668725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 02:59:57.668725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 02:59:57.668725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:57.668725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 02:59:57.668725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:57.668725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:57.668725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:57.668725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:57.668725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:57.668725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:57.669859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:57.669859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:57.669859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:57.669859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:57.669859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:57.669859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:57.669859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 02:59:57.669859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 02:59:57.669859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 02:59:57.669859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 02:59:57.669859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:57.669859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:57.669859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:57.669859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:57.669859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:57.669859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:57.670927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:57.670927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:57.670927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:57.670927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:57.670927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:57.670927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:57.670927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:57.670927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:57.670927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:57.670927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:57.670927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 02:59:57.670927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 02:59:57.670927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 02:59:57.670927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:57.675033 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 316/622] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:57 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:57.676416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8a04e-bc12-4883-81f2-43f39787fec7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:57.679129 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae7043-6a60-441a-bae5-93d376f8bad9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:57.680627 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae7043-6a60-441a-bae5-93d376f8bad9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:57.692611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-d7ae7043-6a60-441a-bae5-93d376f8bad9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:57.692611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:57.692611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:57.692611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:57.692611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:57.692611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:57.692611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:57.692611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:57.692611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:57.692611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:57.692611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:57.692611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:57.692611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 02:59:57.692611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 02:59:57.692611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 02:59:57.693532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 02:59:57.693532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:57.693532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:57.693532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:57.693532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:57.693532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:57.693532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:57.693532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:57.693532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:57.693532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:57.693532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:57.693532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:57.693532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:57.693532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:57.693532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:57.693532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:57.693532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:57.694445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 02:59:57.694445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 02:59:57.694445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 02:59:57.694445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 02:59:57.694445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 02:59:57.694445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 02:59:57.694445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:57.694445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:57.694445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 307/623] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:57.700127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a084f-f8be-458d-8bce-6a07673f8d7f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:57.700127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a084f-f8be-458d-8bce-6a07673f8d7f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:57.700127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a084f-f8be-458d-8bce-6a07673f8d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:57.716144 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-042a084f-f8be-458d-8bce-6a07673f8d7f None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:57.717339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-042a084f-f8be-458d-8bce-6a07673f8d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:57.733524 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:57.770881 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:57.789471 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8a04e-bc12-4883-81f2-43f39787fec7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:57.794948 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 308/624] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 02:59:57.794948 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a084f-f8be-458d-8bce-6a07673f8d7f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:57.835152 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49936d22-8b47-43bb-8181-65028b727a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:57.835859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49936d22-8b47-43bb-8181-65028b727a3e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:57.836209 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49936d22-8b47-43bb-8181-65028b727a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:57.836552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49936d22-8b47-43bb-8181-65028b727a3e None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:57.837396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49936d22-8b47-43bb-8181-65028b727a3e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:57.837772 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49936d22-8b47-43bb-8181-65028b727a3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:57.839903 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49936d22-8b47-43bb-8181-65028b727a3e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:57.863267 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 317/625] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Oct 17 02:59:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:57.864637 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49936d22-8b47-43bb-8181-65028b727a3e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:57.905660 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a084f-f8be-458d-8bce-6a07673f8d7f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:57.942571 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad1276-402e-4f5b-bfe6-3723fca919c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:57.943601 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad1276-402e-4f5b-bfe6-3723fca919c6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:57.944037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad1276-402e-4f5b-bfe6-3723fca919c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:57.944459 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad1276-402e-4f5b-bfe6-3723fca919c6 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:57.945239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad1276-402e-4f5b-bfe6-3723fca919c6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:57.945774 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad1276-402e-4f5b-bfe6-3723fca919c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:57.947322 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad1276-402e-4f5b-bfe6-3723fca919c6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:57.952716 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 309/626] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Oct 17 02:59:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:57.953802 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad1276-402e-4f5b-bfe6-3723fca919c6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:57.970990 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:57.977324 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49936d22-8b47-43bb-8181-65028b727a3e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:58.017113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd3754-8c38-4b67-a020-5f3ed06a46f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:58.018054 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd3754-8c38-4b67-a020-5f3ed06a46f2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:58.018552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd3754-8c38-4b67-a020-5f3ed06a46f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:58.019087 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd3754-8c38-4b67-a020-5f3ed06a46f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:58.020653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd3754-8c38-4b67-a020-5f3ed06a46f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:58.021290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd3754-8c38-4b67-a020-5f3ed06a46f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:58.023169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd3754-8c38-4b67-a020-5f3ed06a46f2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:58.040394 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 318/627] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 02:59:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:58.041268 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd3754-8c38-4b67-a020-5f3ed06a46f2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:58.043943 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:58.073168 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad1276-402e-4f5b-bfe6-3723fca919c6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:58.111672 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1e8c2-3334-4d01-9725-7bcd9c27766a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:58.112517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1e8c2-3334-4d01-9725-7bcd9c27766a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:58.112800 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1e8c2-3334-4d01-9725-7bcd9c27766a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:58.113068 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1e8c2-3334-4d01-9725-7bcd9c27766a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:58.151057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd3754-8c38-4b67-a020-5f3ed06a46f2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:58.187257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1e8c2-3334-4d01-9725-7bcd9c27766a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:58.187789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110e78d-c581-4721-933e-2262d9626c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:58.188423 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110e78d-c581-4721-933e-2262d9626c86 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:58.188704 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110e78d-c581-4721-933e-2262d9626c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:58.188801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1e8c2-3334-4d01-9725-7bcd9c27766a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:58.188974 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110e78d-c581-4721-933e-2262d9626c86 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:58.189595 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110e78d-c581-4721-933e-2262d9626c86 None admin] RBAC: Adding query filter params (type=network) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:58.189893 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110e78d-c581-4721-933e-2262d9626c86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:58.191887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110e78d-c581-4721-933e-2262d9626c86 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:58.196737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 319/628] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Oct 17 02:59:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:58.197240 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110e78d-c581-4721-933e-2262d9626c86 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:58.208404 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:58.271153 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:58.303578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 310/629] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 02:59:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 02:59:58.304721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1e8c2-3334-4d01-9725-7bcd9c27766a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:58.312734 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110e78d-c581-4721-933e-2262d9626c86 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:58.350227 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb24cee-681a-4670-9bd2-dd1ad0a335ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:58.350836 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb24cee-681a-4670-9bd2-dd1ad0a335ff None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:58.351149 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb24cee-681a-4670-9bd2-dd1ad0a335ff None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:58.351421 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb24cee-681a-4670-9bd2-dd1ad0a335ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:58.352146 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb24cee-681a-4670-9bd2-dd1ad0a335ff None admin] RBAC: Adding query filter params (role.id=ea0fd0c10f2f4e529270edbe3525d4c6, scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2, user.id=eceda8b3a8ba4233851d0138d8f4d06a) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:58.352462 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb24cee-681a-4670-9bd2-dd1ad0a335ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:58.353893 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb24cee-681a-4670-9bd2-dd1ad0a335ff None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:58.370891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 320/630] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 02:59:58 2022] GET /identity/v3/role_assignments?user.id=eceda8b3a8ba4233851d0138d8f4d06a&scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2&role.id=ea0fd0c10f2f4e529270edbe3525d4c6 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:58.370891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb24cee-681a-4670-9bd2-dd1ad0a335ff None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:58.446547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1e8c2-3334-4d01-9725-7bcd9c27766a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:58.462990 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:58.477624 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb24cee-681a-4670-9bd2-dd1ad0a335ff None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:58.487941 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f278a-9399-41f7-b469-11dc66a3c135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:58.489700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f278a-9399-41f7-b469-11dc66a3c135 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:58.490276 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f278a-9399-41f7-b469-11dc66a3c135 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:58.490766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f278a-9399-41f7-b469-11dc66a3c135 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:58.491723 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f278a-9399-41f7-b469-11dc66a3c135 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:58.493372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f278a-9399-41f7-b469-11dc66a3c135 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:58.511760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:58.512139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 311/631] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Oct 17 02:59:58 2022] POST /identity/v3/services => generated 269 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:59:58.513128 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f278a-9399-41f7-b469-11dc66a3c135 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:58.534264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62d4e8-f6a0-42b2-804c-6fc3b0fa3891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:58.535361 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62d4e8-f6a0-42b2-804c-6fc3b0fa3891 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:58.536150 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62d4e8-f6a0-42b2-804c-6fc3b0fa3891 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:58.536627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62d4e8-f6a0-42b2-804c-6fc3b0fa3891 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:58.537569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62d4e8-f6a0-42b2-804c-6fc3b0fa3891 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:58.539247 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62d4e8-f6a0-42b2-804c-6fc3b0fa3891 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:58.579033 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 321/632] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 02:59:58 2022] POST /identity/v3/endpoints => generated 337 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 02:59:58.645949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f278a-9399-41f7-b469-11dc66a3c135 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:58.694153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab60175-e603-49f6-90d1-dd76991f8313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:58.694824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab60175-e603-49f6-90d1-dd76991f8313 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:58.695141 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab60175-e603-49f6-90d1-dd76991f8313 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:58.695416 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab60175-e603-49f6-90d1-dd76991f8313 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:58.709005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab60175-e603-49f6-90d1-dd76991f8313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:58.710507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab60175-e603-49f6-90d1-dd76991f8313 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:58.727425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-9ab60175-e603-49f6-90d1-dd76991f8313 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:58.727425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:58.727425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:58.727425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:58.727425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:58.727425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:58.727425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:58.727425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:58.727425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:58.727425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:58.727425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:58.727425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:58.727425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 02:59:58.727425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 02:59:58.727425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 02:59:58.727425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 02:59:58.728441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:58.728441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:58.728441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:58.728441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:58.728441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:58.728441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:58.728441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:58.728441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:58.728441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:58.728441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:58.728441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:58.728441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:58.728441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:58.728441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:58.728441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:58.728441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:58.728441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 02:59:58.729376 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 02:59:58.729376 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 02:59:58.729376 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:58.729376 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 312/633] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 02:59:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:58.735202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62d4e8-f6a0-42b2-804c-6fc3b0fa3891 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:58.851388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62d4e8-f6a0-42b2-804c-6fc3b0fa3891 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:58.896998 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4280d-db7d-4f68-924e-2b7f0b4df105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:58.898005 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4280d-db7d-4f68-924e-2b7f0b4df105 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:58.898523 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4280d-db7d-4f68-924e-2b7f0b4df105 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:58.899057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4280d-db7d-4f68-924e-2b7f0b4df105 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:58.899929 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4280d-db7d-4f68-924e-2b7f0b4df105 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:58.900442 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4280d-db7d-4f68-924e-2b7f0b4df105 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:58.902067 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4280d-db7d-4f68-924e-2b7f0b4df105 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:58.915419 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 322/634] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 02:59:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:58.920800 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab60175-e603-49f6-90d1-dd76991f8313 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:58.962713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:58.992676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c5809-c347-45b0-aaf9-87bb1d3d6ea1 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:58.992873 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c5809-c347-45b0-aaf9-87bb1d3d6ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:58.993144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c5809-c347-45b0-aaf9-87bb1d3d6ea1 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:58.994595 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 323/635] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 02:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 02:59:59.000201 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501fa14-5b82-4f4d-88f6-5b8730dbb659 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:59.000489 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501fa14-5b82-4f4d-88f6-5b8730dbb659 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:59.000767 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501fa14-5b82-4f4d-88f6-5b8730dbb659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:59.026479 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9501fa14-5b82-4f4d-88f6-5b8730dbb659 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 02:59:59.028286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9501fa14-5b82-4f4d-88f6-5b8730dbb659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 02:59:59.061060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab60175-e603-49f6-90d1-dd76991f8313 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:59.074796 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:59.101574 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e4ccb-be6c-4450-a673-ac728f9a4460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:59.102217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e4ccb-be6c-4450-a673-ac728f9a4460 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:59.102494 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e4ccb-be6c-4450-a673-ac728f9a4460 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:59.102760 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e4ccb-be6c-4450-a673-ac728f9a4460 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:59.110484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e4ccb-be6c-4450-a673-ac728f9a4460 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:59.112081 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e4ccb-be6c-4450-a673-ac728f9a4460 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:59.112302 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 324/636] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 02:59:58 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 02:59:59.124507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-093e4ccb-be6c-4450-a673-ac728f9a4460 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:59.124507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:59.124507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:59.124507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:59.124507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:59.124507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:59.124507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:59.124507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:59.124507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:59.124507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:59.124507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:59.124507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:59.124507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 02:59:59.124507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 02:59:59.124507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 02:59:59.125414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 02:59:59.125414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:59.125414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:59.125414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:59.125414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:59.125414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:59.125414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:59.125414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:59.125414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:59.125414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:59.125414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:59.125414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:59.125414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:59.125414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:59.125414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:59.125414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:59.125414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:59.126965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 02:59:59.126965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 02:59:59.126965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 02:59:59.126965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 02:59:59.126965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 02:59:59.126965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 02:59:59.126965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 02:59:59.126965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 02:59:59.126965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 313/637] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 02:59:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 02:59:59.131723 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501fa14-5b82-4f4d-88f6-5b8730dbb659 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:59.234481 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501fa14-5b82-4f4d-88f6-5b8730dbb659 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:59.270908 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60b5d5-3d9c-4dd4-9095-0be1ca0232fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:59.271448 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60b5d5-3d9c-4dd4-9095-0be1ca0232fc None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:59.272059 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60b5d5-3d9c-4dd4-9095-0be1ca0232fc None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:59.272059 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60b5d5-3d9c-4dd4-9095-0be1ca0232fc None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:59.272694 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60b5d5-3d9c-4dd4-9095-0be1ca0232fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:59.272977 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60b5d5-3d9c-4dd4-9095-0be1ca0232fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:59.274411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60b5d5-3d9c-4dd4-9095-0be1ca0232fc None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:59.287369 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 325/638] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 02:59:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:59.293240 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e4ccb-be6c-4450-a673-ac728f9a4460 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:59.382915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:59.393651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60b5d5-3d9c-4dd4-9095-0be1ca0232fc None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:59.406557 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e4ccb-be6c-4450-a673-ac728f9a4460 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:59.427123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:59.443680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d7415-9900-43bd-9ef3-b41268fc4936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:59.444287 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d7415-9900-43bd-9ef3-b41268fc4936 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:59.444568 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d7415-9900-43bd-9ef3-b41268fc4936 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:59.444837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d7415-9900-43bd-9ef3-b41268fc4936 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:59.445562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d7415-9900-43bd-9ef3-b41268fc4936 None admin] RBAC: Adding query filter params (role.id=78dc08989b32455389b21096f5d57e17, scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2, user.id=280d47ed79f14105a83b18b2cac58b60) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:59.445894 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d7415-9900-43bd-9ef3-b41268fc4936 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:59.447321 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d7415-9900-43bd-9ef3-b41268fc4936 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:59.453996 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 314/639] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 02:59:59 2022] GET /identity/v3/role_assignments?user.id=280d47ed79f14105a83b18b2cac58b60&scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2&role.id=78dc08989b32455389b21096f5d57e17 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 02:59:59.518301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60b5d5-3d9c-4dd4-9095-0be1ca0232fc None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:59.559780 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc89f0-aed3-422a-8411-3117a453a752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:59.560429 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc89f0-aed3-422a-8411-3117a453a752 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:59.560706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc89f0-aed3-422a-8411-3117a453a752 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:59.560977 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc89f0-aed3-422a-8411-3117a453a752 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:59.561567 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc89f0-aed3-422a-8411-3117a453a752 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:59.562979 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc89f0-aed3-422a-8411-3117a453a752 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:59.572726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-3cfc89f0-aed3-422a-8411-3117a453a752 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 02:59:59.572726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:59.572726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 02:59:59.572726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 02:59:59.572726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 02:59:59.572726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 02:59:59.572726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 02:59:59.572726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 02:59:59.572726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 02:59:59.572726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:59.572726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 02:59:59.572726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:59.572726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 02:59:59.572726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 02:59:59.572726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 02:59:59.572726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 02:59:59.572726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 02:59:59.573809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 02:59:59.573809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 02:59:59.573809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 02:59:59.573809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 02:59:59.573809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 02:59:59.573809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 02:59:59.573809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 02:59:59.573809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 02:59:59.573809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 02:59:59.573809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 02:59:59.573809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 02:59:59.573809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 02:59:59.573809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 02:59:59.573809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 02:59:59.573809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 02:59:59.573809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 02:59:59.575098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 02:59:59.575098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 02:59:59.575098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 02:59:59.575098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 02:59:59.575098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 02:59:59.575098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 02:59:59.575098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 02:59:59.575098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 02:59:59.575098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 02:59:59.575098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 02:59:59.575098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 02:59:59.575098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 02:59:59.575098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 02:59:59.575098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 02:59:59.575098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 326/640] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Oct 17 02:59:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 02:59:59.582916 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d7415-9900-43bd-9ef3-b41268fc4936 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:59.691920 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d7415-9900-43bd-9ef3-b41268fc4936 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:59.728230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e9899-04fe-42ba-be0b-32b8d7611dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:59.729222 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e9899-04fe-42ba-be0b-32b8d7611dcd None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:59.729724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e9899-04fe-42ba-be0b-32b8d7611dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:59.730211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e9899-04fe-42ba-be0b-32b8d7611dcd None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:59.731090 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e9899-04fe-42ba-be0b-32b8d7611dcd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:59.731622 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e9899-04fe-42ba-be0b-32b8d7611dcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:59.733211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e9899-04fe-42ba-be0b-32b8d7611dcd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:59.744504 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 315/641] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Oct 17 02:59:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 02:59:59.746276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc89f0-aed3-422a-8411-3117a453a752 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 02:59:59.848966 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 02:59:59.855843 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc89f0-aed3-422a-8411-3117a453a752 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 02:59:59.891455 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925baa50-db21-4e76-acdb-de7a03605248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 02:59:59.892069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925baa50-db21-4e76-acdb-de7a03605248 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 02:59:59.892345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925baa50-db21-4e76-acdb-de7a03605248 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 02:59:59.892614 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925baa50-db21-4e76-acdb-de7a03605248 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 02:59:59.893242 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925baa50-db21-4e76-acdb-de7a03605248 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 02:59:59.893543 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925baa50-db21-4e76-acdb-de7a03605248 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 02:59:59.895684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925baa50-db21-4e76-acdb-de7a03605248 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 02:59:59.900850 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 327/642] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Oct 17 02:59:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:00.762626 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadea64-d1ea-45e9-80a6-4fe9bffcf6ee None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:00.763195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadea64-d1ea-45e9-80a6-4fe9bffcf6ee None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:00.763195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eadea64-d1ea-45e9-80a6-4fe9bffcf6ee None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:00.764776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 316/643] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:00.771090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74ad8d-e0b4-4541-9156-0b31fe55d90c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:00.771553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74ad8d-e0b4-4541-9156-0b31fe55d90c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:00.771646 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74ad8d-e0b4-4541-9156-0b31fe55d90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:00.791370 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ab74ad8d-e0b4-4541-9156-0b31fe55d90c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:00.792607 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ab74ad8d-e0b4-4541-9156-0b31fe55d90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:00.860276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 328/644] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:00 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:01.076039 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bd111-52ed-4ccd-9e7d-c6836de0430f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:01.076410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bd111-52ed-4ccd-9e7d-c6836de0430f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:01.076590 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bd111-52ed-4ccd-9e7d-c6836de0430f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:01.078163 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 317/645] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:01.083912 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b0031-8c0b-470d-bb22-fb012f146583 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:01.084200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b0031-8c0b-470d-bb22-fb012f146583 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:01.084479 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b0031-8c0b-470d-bb22-fb012f146583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:01.102727 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f26b0031-8c0b-470d-bb22-fb012f146583 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:01.103958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f26b0031-8c0b-470d-bb22-fb012f146583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:01.135246 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bd111-52ed-4ccd-9e7d-c6836de0430f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:01.149389 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:01.174428 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 329/646] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:01 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:01.246961 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bd111-52ed-4ccd-9e7d-c6836de0430f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:01.287533 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03283220-c3e1-416c-90a1-a38ae9ad858a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:01.288232 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03283220-c3e1-416c-90a1-a38ae9ad858a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:01.288562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03283220-c3e1-416c-90a1-a38ae9ad858a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:01.288841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03283220-c3e1-416c-90a1-a38ae9ad858a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:01.289435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03283220-c3e1-416c-90a1-a38ae9ad858a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:01.290822 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03283220-c3e1-416c-90a1-a38ae9ad858a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:01.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-03283220-c3e1-416c-90a1-a38ae9ad858a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:00:01.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:01.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:00:01.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:00:01.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:00:01.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:00:01.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:00:01.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:01.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:00:01.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:01.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:01.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:01.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:01.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:01.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:01.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:01.300227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:01.303195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:01.303195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:01.303195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:01.303195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:01.303195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:01.303195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:01.303195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:00:01.303195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:00:01.303195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:00:01.303195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:00:01.303195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:01.303195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:01.303195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:01.303195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:01.303195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:01.303195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:01.305035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:01.305035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:01.305035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:01.305035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:01.305035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:01.305035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:01.305035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:01.305035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:01.305035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:01.305035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:01.305035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:00:01.305035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:01.305035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:00:01.305035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:01.305035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 318/647] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Oct 17 03:00:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:01.308298 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b0031-8c0b-470d-bb22-fb012f146583 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:01.336077 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:01.419372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b0031-8c0b-470d-bb22-fb012f146583 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:01.452005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03283220-c3e1-416c-90a1-a38ae9ad858a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:01.455583 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8a667-adc6-4c74-989c-7bddbd17e25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:01.456204 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8a667-adc6-4c74-989c-7bddbd17e25c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:01.456487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8a667-adc6-4c74-989c-7bddbd17e25c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:01.456763 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8a667-adc6-4c74-989c-7bddbd17e25c None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:01.457731 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8a667-adc6-4c74-989c-7bddbd17e25c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:01.458176 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8a667-adc6-4c74-989c-7bddbd17e25c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:01.459579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8a667-adc6-4c74-989c-7bddbd17e25c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:01.464422 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 330/648] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Oct 17 03:00:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:01.469728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8a667-adc6-4c74-989c-7bddbd17e25c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:01.548158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:01.577132 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03283220-c3e1-416c-90a1-a38ae9ad858a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:01.577853 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:01.596067 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8a667-adc6-4c74-989c-7bddbd17e25c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:01.617216 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f9204-4e10-44d2-99b1-07faeae4666b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:01.617845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f9204-4e10-44d2-99b1-07faeae4666b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:01.618121 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f9204-4e10-44d2-99b1-07faeae4666b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:01.618390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f9204-4e10-44d2-99b1-07faeae4666b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:01.619001 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f9204-4e10-44d2-99b1-07faeae4666b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:01.620924 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f9204-4e10-44d2-99b1-07faeae4666b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:01.633886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-644f9204-4e10-44d2-99b1-07faeae4666b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 03:00:01.633886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:01.633886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:00:01.633886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:00:01.633886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:00:01.633886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:00:01.633886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:00:01.633886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:01.633886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 03:00:01.633886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:01.633886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:01.633886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:01.633886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:01.633886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:01.633886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:01.633886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:01.635180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:01.635180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:01.635180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:01.635180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:01.635180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:01.635180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:01.635180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:01.635180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:00:01.635180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:00:01.635180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:00:01.635180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:00:01.635180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:01.635180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:01.635180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:01.635180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:01.635180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:01.635180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:01.636110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:01.636110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:01.636110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:01.636110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:01.636110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:01.636110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:01.636110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:01.636110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:01.636110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:01.636110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:01.636110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:00:01.636110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:01.636110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 03:00:01.636110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:01.639815 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f7277-ab4f-4c3b-bde8-531366cafea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:01.639957 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 319/649] 10.222.0.27 () {56 vars in 1132 bytes} [Mon Oct 17 03:00:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:00:01.640725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f9204-4e10-44d2-99b1-07faeae4666b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:01.640992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f7277-ab4f-4c3b-bde8-531366cafea9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:01.641598 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f7277-ab4f-4c3b-bde8-531366cafea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:01.642075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f7277-ab4f-4c3b-bde8-531366cafea9 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:01.642948 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f7277-ab4f-4c3b-bde8-531366cafea9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:01.643552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f7277-ab4f-4c3b-bde8-531366cafea9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:01.645813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f7277-ab4f-4c3b-bde8-531366cafea9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:01.651386 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 331/650] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Oct 17 03:00:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:01.766535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f9204-4e10-44d2-99b1-07faeae4666b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:01.792195 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cab0d-c875-412e-8d0c-f83452e4337b None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:01.792716 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cab0d-c875-412e-8d0c-f83452e4337b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:01.792793 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453cab0d-c875-412e-8d0c-f83452e4337b None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:01.794361 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 332/651] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:00:01.799685 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995fc62-ff25-4864-8c6b-375368e52c4d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:01.799972 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995fc62-ff25-4864-8c6b-375368e52c4d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:01.800249 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995fc62-ff25-4864-8c6b-375368e52c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:01.811876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b629cb-993b-42b7-821c-f95ad51a24c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:01.812503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b629cb-993b-42b7-821c-f95ad51a24c8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:01.812781 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b629cb-993b-42b7-821c-f95ad51a24c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:01.813051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b629cb-993b-42b7-821c-f95ad51a24c8 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:01.813699 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b629cb-993b-42b7-821c-f95ad51a24c8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:01.814004 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b629cb-993b-42b7-821c-f95ad51a24c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:01.815452 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b629cb-993b-42b7-821c-f95ad51a24c8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:01.822098 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 320/652] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Oct 17 03:00:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:01.827898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b629cb-993b-42b7-821c-f95ad51a24c8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:01.833153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:01.837244 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1995fc62-ff25-4864-8c6b-375368e52c4d None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:01.838492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1995fc62-ff25-4864-8c6b-375368e52c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:01.935890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 333/653] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:01 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:00:01.954821 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b629cb-993b-42b7-821c-f95ad51a24c8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:01.993758 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951bbb7e-a01d-4516-bd35-1d2b4d9662f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:01.994395 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bbb7e-a01d-4516-bd35-1d2b4d9662f7 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:01.994680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bbb7e-a01d-4516-bd35-1d2b4d9662f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:01.994990 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951bbb7e-a01d-4516-bd35-1d2b4d9662f7 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:01.995654 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bbb7e-a01d-4516-bd35-1d2b4d9662f7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:01.995958 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bbb7e-a01d-4516-bd35-1d2b4d9662f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:01.997391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951bbb7e-a01d-4516-bd35-1d2b4d9662f7 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:02.003784 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 321/654] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Oct 17 03:00:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:02.010201 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995fc62-ff25-4864-8c6b-375368e52c4d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:02.094255 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:02.105013 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc161233-ab66-4e8c-8e33-950720526f87 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:02.105301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc161233-ab66-4e8c-8e33-950720526f87 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:02.105935 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc161233-ab66-4e8c-8e33-950720526f87 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:02.108237 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 322/655] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:00:02.112086 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123ecb6-c01b-4c0c-aa23-eb500be1125c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:02.112378 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123ecb6-c01b-4c0c-aa23-eb500be1125c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:02.112661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123ecb6-c01b-4c0c-aa23-eb500be1125c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:02.120123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995fc62-ff25-4864-8c6b-375368e52c4d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:02.134419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3123ecb6-c01b-4c0c-aa23-eb500be1125c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:02.136296 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3123ecb6-c01b-4c0c-aa23-eb500be1125c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:02.155480 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df6f48-a1c7-48c0-820b-d9512869c16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:02.156408 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df6f48-a1c7-48c0-820b-d9512869c16f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:02.156904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df6f48-a1c7-48c0-820b-d9512869c16f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:02.157507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0df6f48-a1c7-48c0-820b-d9512869c16f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:02.158410 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df6f48-a1c7-48c0-820b-d9512869c16f None admin] RBAC: Adding query filter params (interface=public, service_id=6d835e14ac2f4ff79daaa66ab4c4bc1f, region_id=RegionOne) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:02.158912 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df6f48-a1c7-48c0-820b-d9512869c16f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:02.160585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0df6f48-a1c7-48c0-820b-d9512869c16f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:02.166297 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 334/656] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Oct 17 03:00:02 2022] GET /identity/v3/endpoints?service_id=6d835e14ac2f4ff79daaa66ab4c4bc1f&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:02.171880 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df6f48-a1c7-48c0-820b-d9512869c16f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:02.208086 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:02.219721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 323/657] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:02 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:00:02.220832 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123ecb6-c01b-4c0c-aa23-eb500be1125c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:02.283907 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0df6f48-a1c7-48c0-820b-d9512869c16f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:02.321890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391eb67-9107-494f-99b0-87f076269109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:02.322966 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391eb67-9107-494f-99b0-87f076269109 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:02.323333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391eb67-9107-494f-99b0-87f076269109 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:02.323693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391eb67-9107-494f-99b0-87f076269109 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:02.324618 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391eb67-9107-494f-99b0-87f076269109 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:02.326561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391eb67-9107-494f-99b0-87f076269109 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:02.332472 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123ecb6-c01b-4c0c-aa23-eb500be1125c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:02.334899 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 335/658] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Oct 17 03:00:02 2022] GET /identity/v3/services => generated 1029 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:00:02.382764 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442d3bb-bbce-4837-8246-faccb6fc2ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:02.383436 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442d3bb-bbce-4837-8246-faccb6fc2ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:02.383715 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442d3bb-bbce-4837-8246-faccb6fc2ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:02.383986 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442d3bb-bbce-4837-8246-faccb6fc2ac1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:02.397269 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442d3bb-bbce-4837-8246-faccb6fc2ac1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:00:02.402218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2442d3bb-bbce-4837-8246-faccb6fc2ac1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 03:00:02.402218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:02.402218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:02.402218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:02.402218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:02.402218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:02.402218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:02.402218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:02.402218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:02.402218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:02.402218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:02.402218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:02.402218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:00:02.402218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:00:02.402218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:00:02.402218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:00:02.403272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:00:02.403272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:00:02.403272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:00:02.403272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:00:02.403272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:02.403272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:02.403272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:02.403272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:02.403272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:02.403272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:02.403272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:02.403272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:02.403272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:02.403272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:02.403272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:02.403272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:02.404789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:02.404789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:02.404789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:02.404789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:02.404789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:00:02.404789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:00:02.404789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:00:02.404789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:00:02.404789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:00:02.404789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:00:02.404789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 03:00:02.404789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:02.404789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 324/659] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Oct 17 03:00:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:02.409410 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391eb67-9107-494f-99b0-87f076269109 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:02.419672 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:02.500196 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442d3bb-bbce-4837-8246-faccb6fc2ac1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:02.528833 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391eb67-9107-494f-99b0-87f076269109 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:02.569354 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:02.572279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796e375-cbf7-49f3-8fc6-e225b5080f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:02.572897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796e375-cbf7-49f3-8fc6-e225b5080f7b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:02.573184 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796e375-cbf7-49f3-8fc6-e225b5080f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:02.573459 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796e375-cbf7-49f3-8fc6-e225b5080f7b None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:02.574130 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796e375-cbf7-49f3-8fc6-e225b5080f7b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:02.574557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796e375-cbf7-49f3-8fc6-e225b5080f7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:02.575848 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796e375-cbf7-49f3-8fc6-e225b5080f7b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:02.589388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 336/660] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Oct 17 03:00:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:02.591688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796e375-cbf7-49f3-8fc6-e225b5080f7b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:02.631907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442d3bb-bbce-4837-8246-faccb6fc2ac1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:02.667532 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:02.681285 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ef511-d8ae-4938-8729-91ed54150c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:02.682050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ef511-d8ae-4938-8729-91ed54150c5b None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:02.682439 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ef511-d8ae-4938-8729-91ed54150c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:02.683064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ef511-d8ae-4938-8729-91ed54150c5b None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:02.684631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ef511-d8ae-4938-8729-91ed54150c5b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:02.686024 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ef511-d8ae-4938-8729-91ed54150c5b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:02.704517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 325/661] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Oct 17 03:00:02 2022] POST /identity/v3/services => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:00:02.710817 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796e375-cbf7-49f3-8fc6-e225b5080f7b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:02.756551 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86144811-14a0-47bd-8c40-a6f154f5c0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:02.757562 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86144811-14a0-47bd-8c40-a6f154f5c0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:02.758062 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86144811-14a0-47bd-8c40-a6f154f5c0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:02.758542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86144811-14a0-47bd-8c40-a6f154f5c0a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:02.771915 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86144811-14a0-47bd-8c40-a6f154f5c0a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:02.773824 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86144811-14a0-47bd-8c40-a6f154f5c0a0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:02.790238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-86144811-14a0-47bd-8c40-a6f154f5c0a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:02.790238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:02.790238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:02.790238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:02.790238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:02.790238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:02.790238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:02.790238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:02.790238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:02.790238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:02.790238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:02.790238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:02.790238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:00:02.790238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:00:02.790238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:00:02.790238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:00:02.791290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:02.791290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:02.791290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:02.791290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:02.791290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:02.791290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:02.791290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:02.791290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:02.791290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:02.791290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:02.791290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:02.791290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:02.791290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:02.791290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:02.791290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:02.791290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:02.791290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:00:02.792212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:00:02.792212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:02.792212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:02.794994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 337/662] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:00:02.806933 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ef511-d8ae-4938-8729-91ed54150c5b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:02.919583 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ef511-d8ae-4938-8729-91ed54150c5b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:02.939928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:02.961489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af9f22-aa09-495f-b422-d13848e80dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:02.962108 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af9f22-aa09-495f-b422-d13848e80dca None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:02.962391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af9f22-aa09-495f-b422-d13848e80dca None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:02.962660 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70af9f22-aa09-495f-b422-d13848e80dca None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:02.963612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af9f22-aa09-495f-b422-d13848e80dca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:02.963612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af9f22-aa09-495f-b422-d13848e80dca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:02.964981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70af9f22-aa09-495f-b422-d13848e80dca None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:02.978971 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 326/663] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:02.986992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86144811-14a0-47bd-8c40-a6f154f5c0a0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:03.092196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86144811-14a0-47bd-8c40-a6f154f5c0a0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:03.109840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:03.128332 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef20a6a-47ae-428f-aa0c-4c78a19b5165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:03.129545 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef20a6a-47ae-428f-aa0c-4c78a19b5165 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:03.130392 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef20a6a-47ae-428f-aa0c-4c78a19b5165 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:03.130673 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef20a6a-47ae-428f-aa0c-4c78a19b5165 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:03.139568 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef20a6a-47ae-428f-aa0c-4c78a19b5165 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:03.141024 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef20a6a-47ae-428f-aa0c-4c78a19b5165 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:03.153358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-eef20a6a-47ae-428f-aa0c-4c78a19b5165 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 03:00:03.153358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:03.153358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:03.153358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:03.153358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:03.153358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:03.153358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:03.153358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:03.153358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:03.153358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:03.153358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:03.153358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:03.153358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:00:03.153358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:00:03.153358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:00:03.153358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:03.154345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:03.155390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:03.155390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:03.155390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:00:03.155390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:00:03.155390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:00:03.155390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:00:03.155390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:00:03.155390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:00:03.155390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 03:00:03.155390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:03.155390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 338/664] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Oct 17 03:00:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:03.161537 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af9f22-aa09-495f-b422-d13848e80dca None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:03.274555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70af9f22-aa09-495f-b422-d13848e80dca None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:03.312366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff0a30-1b33-4620-914f-478759db4f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:03.312973 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff0a30-1b33-4620-914f-478759db4f63 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:03.313252 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff0a30-1b33-4620-914f-478759db4f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:03.313522 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff0a30-1b33-4620-914f-478759db4f63 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:03.314248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff0a30-1b33-4620-914f-478759db4f63 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:03.314553 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff0a30-1b33-4620-914f-478759db4f63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:03.316030 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff0a30-1b33-4620-914f-478759db4f63 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:03.335941 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 327/665] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Oct 17 03:00:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:03.341773 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef20a6a-47ae-428f-aa0c-4c78a19b5165 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:03.449157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef20a6a-47ae-428f-aa0c-4c78a19b5165 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:03.484161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11796fb-27a5-4f3a-89ae-0cbe87fb9fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:03.484799 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11796fb-27a5-4f3a-89ae-0cbe87fb9fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:03.485082 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11796fb-27a5-4f3a-89ae-0cbe87fb9fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:03.485356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11796fb-27a5-4f3a-89ae-0cbe87fb9fb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:03.497144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11796fb-27a5-4f3a-89ae-0cbe87fb9fb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:03.498609 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11796fb-27a5-4f3a-89ae-0cbe87fb9fb6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:03.513710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-f11796fb-27a5-4f3a-89ae-0cbe87fb9fb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:03.513710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:03.513710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:03.513710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:03.513710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:03.513710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:03.513710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:03.513710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:03.513710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:03.513710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:03.513710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:03.513710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:03.513710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:00:03.513710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:00:03.513710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:00:03.513710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:00:03.514764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:03.514764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:03.514764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:03.514764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:03.514764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:03.514764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:03.514764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:03.514764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:03.514764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:03.514764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:03.514764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:03.514764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:03.514764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:03.514764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:03.514764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:03.514764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:03.514764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:00:03.515728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:00:03.515728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:03.515728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:03.515728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 339/666] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:03.520988 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff0a30-1b33-4620-914f-478759db4f63 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:03.556502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:03.626411 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff0a30-1b33-4620-914f-478759db4f63 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:03.664048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ee44d-ec41-4521-aa06-c3cb8ed2dd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:03.664672 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ee44d-ec41-4521-aa06-c3cb8ed2dd5a None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:03.664948 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ee44d-ec41-4521-aa06-c3cb8ed2dd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:03.665217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ee44d-ec41-4521-aa06-c3cb8ed2dd5a None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:03.665872 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ee44d-ec41-4521-aa06-c3cb8ed2dd5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:03.666171 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ee44d-ec41-4521-aa06-c3cb8ed2dd5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:03.667575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ee44d-ec41-4521-aa06-c3cb8ed2dd5a None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:03.680137 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 328/667] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:03.685608 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11796fb-27a5-4f3a-89ae-0cbe87fb9fb6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:03.715867 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:03.791900 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11796fb-27a5-4f3a-89ae-0cbe87fb9fb6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:03.826481 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a8e57-9b28-4259-86f1-7b48445ca59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:03.827231 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a8e57-9b28-4259-86f1-7b48445ca59e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:03.827419 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a8e57-9b28-4259-86f1-7b48445ca59e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:03.827703 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a8e57-9b28-4259-86f1-7b48445ca59e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:03.834487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a8e57-9b28-4259-86f1-7b48445ca59e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:03.836011 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a8e57-9b28-4259-86f1-7b48445ca59e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:03.847084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-b66a8e57-9b28-4259-86f1-7b48445ca59e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:00:03.847084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:03.847084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:03.847084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:03.847084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:03.847084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:03.847084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:03.847084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:03.847084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:03.847084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:03.847084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:03.847084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:03.847084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:00:03.847084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:00:03.847084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:00:03.848548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:00:03.848548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:03.848548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:03.848548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:03.848548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:03.848548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:03.848548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:03.848548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:03.848548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:03.848548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:03.848548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:03.848548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:03.848548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:03.848548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:03.848548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:03.848548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:03.848548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:03.849464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:00:03.849464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:00:03.849464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:00:03.849464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:00:03.849464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:00:03.849464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:00:03.849464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:00:03.849464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:03.849464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 340/668] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 03:00:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:03.856138 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ee44d-ec41-4521-aa06-c3cb8ed2dd5a None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:03.960965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:03.968499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ee44d-ec41-4521-aa06-c3cb8ed2dd5a None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:04.004432 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16778a8e-75e5-4643-8f30-bed36a2130a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:04.005492 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16778a8e-75e5-4643-8f30-bed36a2130a6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:04.006281 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16778a8e-75e5-4643-8f30-bed36a2130a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:04.006788 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16778a8e-75e5-4643-8f30-bed36a2130a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:04.007703 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16778a8e-75e5-4643-8f30-bed36a2130a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:04.008205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16778a8e-75e5-4643-8f30-bed36a2130a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:04.009878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16778a8e-75e5-4643-8f30-bed36a2130a6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:04.026578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 329/669] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 03:00:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:04.028391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d14fb1-441d-40fa-be1f-6d00b9710c28 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:04.028741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16778a8e-75e5-4643-8f30-bed36a2130a6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:04.029265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d14fb1-441d-40fa-be1f-6d00b9710c28 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:04.029748 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d14fb1-441d-40fa-be1f-6d00b9710c28 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:04.031653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 341/670] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:04.037582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50596761-1425-4b8c-b27a-584845803d6a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:04.037875 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50596761-1425-4b8c-b27a-584845803d6a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:04.038156 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50596761-1425-4b8c-b27a-584845803d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:04.056942 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-50596761-1425-4b8c-b27a-584845803d6a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:04.058244 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-50596761-1425-4b8c-b27a-584845803d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:04.141772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16778a8e-75e5-4643-8f30-bed36a2130a6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:04.148353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 342/671] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:04.180940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5e01f-f111-409b-a6fb-3bb9c8c0b7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:04.181601 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5e01f-f111-409b-a6fb-3bb9c8c0b7f1 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:04.181879 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5e01f-f111-409b-a6fb-3bb9c8c0b7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:04.182205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5e01f-f111-409b-a6fb-3bb9c8c0b7f1 None admin] PATH_INFO: `/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/280d47ed79f14105a83b18b2cac58b60/roles/355fbb07456348ac8f6ae42b3dd96690` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:04.184297 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e5e01f-f111-409b-a6fb-3bb9c8c0b7f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=89b0ee95955f465eb9ee5b8ae337d9c2, user_id=280d47ed79f14105a83b18b2cac58b60, role_id=355fbb07456348ac8f6ae42b3dd96690)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:04.185721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e5e01f-f111-409b-a6fb-3bb9c8c0b7f1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:04.200493 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 330/672] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 03:00:04 2022] PUT /identity/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/280d47ed79f14105a83b18b2cac58b60/roles/355fbb07456348ac8f6ae42b3dd96690 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:00:04.423698 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50596761-1425-4b8c-b27a-584845803d6a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:04.444127 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:04.535747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50596761-1425-4b8c-b27a-584845803d6a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:04.573083 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696db7a-fcbf-46e8-bf18-ad34d1697573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:04.574116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696db7a-fcbf-46e8-bf18-ad34d1697573 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:04.574646 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696db7a-fcbf-46e8-bf18-ad34d1697573 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:04.575168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696db7a-fcbf-46e8-bf18-ad34d1697573 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:04.576057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696db7a-fcbf-46e8-bf18-ad34d1697573 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:04.577663 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696db7a-fcbf-46e8-bf18-ad34d1697573 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:04.595126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 343/673] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Oct 17 03:00:04 2022] POST /identity/v3/services => generated 241 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:00:04.742013 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b11f9-e4b7-4da7-ad50-c83c28bd9848 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:04.742729 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b11f9-e4b7-4da7-ad50-c83c28bd9848 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:04.743262 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b11f9-e4b7-4da7-ad50-c83c28bd9848 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:04.745206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 331/674] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:04.751321 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48653bdc-0d60-4444-be1f-b1b1d76a0504 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:04.751923 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48653bdc-0d60-4444-be1f-b1b1d76a0504 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:04.752429 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48653bdc-0d60-4444-be1f-b1b1d76a0504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:04.772992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-48653bdc-0d60-4444-be1f-b1b1d76a0504 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:04.774537 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-48653bdc-0d60-4444-be1f-b1b1d76a0504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:04.860743 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 344/675] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:04.957771 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cc13f-4581-41c3-a79d-dcd874f0fd43 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:04.958482 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cc13f-4581-41c3-a79d-dcd874f0fd43 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:04.958482 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cc13f-4581-41c3-a79d-dcd874f0fd43 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:04.959982 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 332/676] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:04.965314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100f60d-1ee9-45bd-94e8-85a954e42471 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:04.965612 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100f60d-1ee9-45bd-94e8-85a954e42471 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:04.965891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100f60d-1ee9-45bd-94e8-85a954e42471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:04.973432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:04.986489 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4100f60d-1ee9-45bd-94e8-85a954e42471 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:04.989225 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4100f60d-1ee9-45bd-94e8-85a954e42471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:05.060366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 345/677] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:05.144033 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cc13f-4581-41c3-a79d-dcd874f0fd43 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:05.161757 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:05.251100 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cc13f-4581-41c3-a79d-dcd874f0fd43 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:05.291959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a588311-7b58-4d76-a6c6-86ef0eae992f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:05.292596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a588311-7b58-4d76-a6c6-86ef0eae992f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:05.292874 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a588311-7b58-4d76-a6c6-86ef0eae992f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:05.293144 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a588311-7b58-4d76-a6c6-86ef0eae992f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:05.293733 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a588311-7b58-4d76-a6c6-86ef0eae992f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:05.295201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a588311-7b58-4d76-a6c6-86ef0eae992f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:05.304521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-7a588311-7b58-4d76-a6c6-86ef0eae992f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 03:00:05.304521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:05.304521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:00:05.304521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:00:05.304521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:00:05.304521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:00:05.304521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:00:05.304521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:05.304521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 03:00:05.304521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:05.304521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:05.304521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:05.304521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:05.304521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:05.304521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:05.304521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:05.305491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:05.305491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:05.305491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:05.305491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:05.305491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:05.305491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:05.305491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:05.305491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:00:05.305491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:00:05.305491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:00:05.305491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:00:05.305491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:05.305491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:05.305491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:05.305491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:05.305491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:05.305491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:05.306390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:05.306390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:05.306390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:05.306390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:05.306390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:05.306390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:05.306390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:05.306390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:05.306390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:05.306390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:05.306390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:00:05.306390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:05.306390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 03:00:05.306390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:05.306390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 333/678] 10.222.0.27 () {56 vars in 1132 bytes} [Mon Oct 17 03:00:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:05.311716 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100f60d-1ee9-45bd-94e8-85a954e42471 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:05.328218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a588311-7b58-4d76-a6c6-86ef0eae992f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:05.416879 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:05.418437 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100f60d-1ee9-45bd-94e8-85a954e42471 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:05.444504 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a588311-7b58-4d76-a6c6-86ef0eae992f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:05.455980 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3e5c8-ba4b-4213-b612-c6f23292b3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:05.456680 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3e5c8-ba4b-4213-b612-c6f23292b3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:05.457023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3e5c8-ba4b-4213-b612-c6f23292b3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:05.457437 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3e5c8-ba4b-4213-b612-c6f23292b3f0 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:05.458240 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3e5c8-ba4b-4213-b612-c6f23292b3f0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:05.458611 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3e5c8-ba4b-4213-b612-c6f23292b3f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:05.460082 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3e5c8-ba4b-4213-b612-c6f23292b3f0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:05.461477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:05.466905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 346/679] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Oct 17 03:00:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:05.472269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3e5c8-ba4b-4213-b612-c6f23292b3f0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:05.495483 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff922364-e689-48e6-9a2f-9fc92f4bbbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:05.496756 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff922364-e689-48e6-9a2f-9fc92f4bbbd9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:05.497126 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff922364-e689-48e6-9a2f-9fc92f4bbbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:05.497471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff922364-e689-48e6-9a2f-9fc92f4bbbd9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:05.498411 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff922364-e689-48e6-9a2f-9fc92f4bbbd9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:05.500173 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff922364-e689-48e6-9a2f-9fc92f4bbbd9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:05.513898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-ff922364-e689-48e6-9a2f-9fc92f4bbbd9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 03:00:05.513898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:05.513898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:00:05.513898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:00:05.513898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:00:05.513898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:00:05.513898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:00:05.513898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:05.513898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 03:00:05.513898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:05.513898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:05.513898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:05.513898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:05.513898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:05.513898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:05.513898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:05.513898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:05.514979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:05.514979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:05.514979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:05.514979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:05.514979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:05.514979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:05.514979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:00:05.514979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:00:05.514979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:00:05.514979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:00:05.514979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:05.514979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:05.514979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:05.514979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:05.514979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:05.514979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:05.515824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:05.515824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:05.515824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:05.515824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:05.515824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:05.515824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:05.515824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:05.515824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:05.515824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:05.515824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:05.515824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:00:05.515824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:05.515824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 03:00:05.515824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:05.521210 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 334/680] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Oct 17 03:00:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:00:05.522147 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff922364-e689-48e6-9a2f-9fc92f4bbbd9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:05.590601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3e5c8-ba4b-4213-b612-c6f23292b3f0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:05.633044 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3051d8d-edfb-480b-ac31-c1a8bef67305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:05.634146 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3051d8d-edfb-480b-ac31-c1a8bef67305 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:05.634501 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3051d8d-edfb-480b-ac31-c1a8bef67305 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:05.634854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3051d8d-edfb-480b-ac31-c1a8bef67305 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:05.635883 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3051d8d-edfb-480b-ac31-c1a8bef67305 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:05.636260 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3051d8d-edfb-480b-ac31-c1a8bef67305 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:05.638195 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3051d8d-edfb-480b-ac31-c1a8bef67305 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:05.647011 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 347/681] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Oct 17 03:00:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:05.650754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3051d8d-edfb-480b-ac31-c1a8bef67305 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:05.658928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff922364-e689-48e6-9a2f-9fc92f4bbbd9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:05.694374 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3527d-3101-466a-b319-b60d891c31b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:05.695090 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3527d-3101-466a-b319-b60d891c31b4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:05.695500 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3527d-3101-466a-b319-b60d891c31b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:05.695865 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a3527d-3101-466a-b319-b60d891c31b4 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:05.696625 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3527d-3101-466a-b319-b60d891c31b4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:05.697018 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3527d-3101-466a-b319-b60d891c31b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:05.698943 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a3527d-3101-466a-b319-b60d891c31b4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:05.708913 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 335/682] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Oct 17 03:00:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:05.709985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3527d-3101-466a-b319-b60d891c31b4 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:05.719212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:05.733423 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:05.790108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3051d8d-edfb-480b-ac31-c1a8bef67305 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:05.822156 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a3527d-3101-466a-b319-b60d891c31b4 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:05.835645 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10807ce2-c710-4f63-9d3a-5335ea000bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:05.836748 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10807ce2-c710-4f63-9d3a-5335ea000bb0 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:05.837184 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10807ce2-c710-4f63-9d3a-5335ea000bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:05.838087 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10807ce2-c710-4f63-9d3a-5335ea000bb0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:05.839214 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10807ce2-c710-4f63-9d3a-5335ea000bb0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:05.840998 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10807ce2-c710-4f63-9d3a-5335ea000bb0 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:05.860743 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfd486-eba5-4be8-9128-f93fff0b0a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:05.861638 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfd486-eba5-4be8-9128-f93fff0b0a38 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:05.862375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfd486-eba5-4be8-9128-f93fff0b0a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:05.862915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfd486-eba5-4be8-9128-f93fff0b0a38 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:05.864356 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dfd486-eba5-4be8-9128-f93fff0b0a38 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:05.865038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dfd486-eba5-4be8-9128-f93fff0b0a38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:05.871681 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 348/683] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:00:05 2022] POST /identity/v3/endpoints => generated 347 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:00:05.872204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dfd486-eba5-4be8-9128-f93fff0b0a38 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:05.883009 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 336/684] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Oct 17 03:00:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:05.884812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10807ce2-c710-4f63-9d3a-5335ea000bb0 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:06.009695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10807ce2-c710-4f63-9d3a-5335ea000bb0 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:06.031240 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:06.053524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071a3f3-8ca2-486e-b78f-d9c8bdfb9b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:06.054168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071a3f3-8ca2-486e-b78f-d9c8bdfb9b19 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:06.054812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071a3f3-8ca2-486e-b78f-d9c8bdfb9b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:06.055734 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071a3f3-8ca2-486e-b78f-d9c8bdfb9b19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:06.056515 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071a3f3-8ca2-486e-b78f-d9c8bdfb9b19 None admin] RBAC: Adding query filter params (interface=public, service_id=134fd7198d564a66a137a72d50218bd8, region_id=RegionOne) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:06.056857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071a3f3-8ca2-486e-b78f-d9c8bdfb9b19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:06.058613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071a3f3-8ca2-486e-b78f-d9c8bdfb9b19 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:06.065083 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 349/685] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Oct 17 03:00:05 2022] GET /identity/v3/endpoints?service_id=134fd7198d564a66a137a72d50218bd8&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:06.071035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfd486-eba5-4be8-9128-f93fff0b0a38 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:06.147865 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:06.185194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfd486-eba5-4be8-9128-f93fff0b0a38 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:06.226169 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec71f6-3109-43b6-98d9-46d26a4140fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:06.226793 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec71f6-3109-43b6-98d9-46d26a4140fa None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:06.227102 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec71f6-3109-43b6-98d9-46d26a4140fa None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:06.227380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec71f6-3109-43b6-98d9-46d26a4140fa None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:06.227998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec71f6-3109-43b6-98d9-46d26a4140fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:06.229375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec71f6-3109-43b6-98d9-46d26a4140fa None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:06.235837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 337/686] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Oct 17 03:00:06 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:00:06.455042 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82488142-6361-48be-95bb-942a42e46113 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:06.455408 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82488142-6361-48be-95bb-942a42e46113 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:06.455611 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82488142-6361-48be-95bb-942a42e46113 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:06.457762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 350/687] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:06.463237 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85746e58-9352-44fb-b653-beafd85753c5 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:06.463532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85746e58-9352-44fb-b653-beafd85753c5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:06.463814 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85746e58-9352-44fb-b653-beafd85753c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:06.482305 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-85746e58-9352-44fb-b653-beafd85753c5 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:06.483542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-85746e58-9352-44fb-b653-beafd85753c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:06.560601 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 338/688] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:06.831220 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82488142-6361-48be-95bb-942a42e46113 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:06.922215 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889f8f5-e7fd-4bc1-a5e9-ccc8448fd4e1 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:06.922525 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889f8f5-e7fd-4bc1-a5e9-ccc8448fd4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:06.922778 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889f8f5-e7fd-4bc1-a5e9-ccc8448fd4e1 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:06.926934 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 339/689] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:06.931458 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf559c1c-6aa0-4951-8595-7c85eb6eb0ec None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:06.931856 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf559c1c-6aa0-4951-8595-7c85eb6eb0ec None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:06.932252 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf559c1c-6aa0-4951-8595-7c85eb6eb0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:06.949783 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82488142-6361-48be-95bb-942a42e46113 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:06.951514 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-bf559c1c-6aa0-4951-8595-7c85eb6eb0ec None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:06.952802 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bf559c1c-6aa0-4951-8595-7c85eb6eb0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:06.986557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c5fc8-6dc8-4c0d-bfc3-e8788a760477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:06.987805 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c5fc8-6dc8-4c0d-bfc3-e8788a760477 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:06.988356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c5fc8-6dc8-4c0d-bfc3-e8788a760477 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:06.989027 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c5fc8-6dc8-4c0d-bfc3-e8788a760477 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:07.000827 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c5fc8-6dc8-4c0d-bfc3-e8788a760477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:00:07.007259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-850c5fc8-6dc8-4c0d-bfc3-e8788a760477 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 03:00:07.007259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:07.007259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:07.007259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:07.007259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:07.007259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:07.007259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:07.007259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:07.007259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:07.007259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:07.007259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:07.007259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:07.007259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:00:07.007259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:00:07.007259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:00:07.007259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:00:07.008414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:00:07.008414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:00:07.008414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:00:07.008414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:00:07.008414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:07.008414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:07.008414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:07.008414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:07.008414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:07.008414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:07.008414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:07.008414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:07.008414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:07.008414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:07.008414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:07.008414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:07.009467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:07.009467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:07.009467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:07.009467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:07.009467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:00:07.009467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:00:07.009467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:00:07.009467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:00:07.009467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:00:07.009467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:00:07.009467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 03:00:07.009467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:07.011863 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 351/690] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Oct 17 03:00:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:07.017827 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c5fc8-6dc8-4c0d-bfc3-e8788a760477 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:07.026888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:07.047246 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 340/691] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:00:07.075374 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:07.124664 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c5fc8-6dc8-4c0d-bfc3-e8788a760477 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:07.163253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1e766-909d-4334-868c-738753b77444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:07.164144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1e766-909d-4334-868c-738753b77444 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:07.164578 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1e766-909d-4334-868c-738753b77444 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:07.165001 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1e766-909d-4334-868c-738753b77444 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:07.165838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a1e766-909d-4334-868c-738753b77444 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:07.166290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a1e766-909d-4334-868c-738753b77444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:07.167825 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a1e766-909d-4334-868c-738753b77444 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:07.177282 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 352/692] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Oct 17 03:00:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:07.183040 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf559c1c-6aa0-4951-8595-7c85eb6eb0ec None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:07.284859 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf559c1c-6aa0-4951-8595-7c85eb6eb0ec None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:07.320817 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b46e3-1ceb-4796-ae0d-755ad6727655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:07.321450 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b46e3-1ceb-4796-ae0d-755ad6727655 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:07.321735 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b46e3-1ceb-4796-ae0d-755ad6727655 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:07.322010 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b46e3-1ceb-4796-ae0d-755ad6727655 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:07.332958 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b46e3-1ceb-4796-ae0d-755ad6727655 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:07.334410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b46e3-1ceb-4796-ae0d-755ad6727655 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:07.346903 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1e766-909d-4334-868c-738753b77444 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:07.349523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-c54b46e3-1ceb-4796-ae0d-755ad6727655 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:07.349523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:07.349523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:07.349523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:07.349523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:07.349523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:07.349523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:07.349523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:07.349523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:07.349523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:07.349523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:07.349523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:07.349523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:00:07.349523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:00:07.349523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:00:07.349523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:00:07.350477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:07.350477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:07.350477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:07.350477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:07.350477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:07.350477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:07.350477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:07.350477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:07.350477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:07.350477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:07.350477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:07.350477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:07.350477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:07.350477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:07.350477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:07.350477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:07.350477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:00:07.351424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:00:07.351424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:07.351424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:07.354970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 341/693] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:00:07.356428 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b46e3-1ceb-4796-ae0d-755ad6727655 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:07.404795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:07.459301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b46e3-1ceb-4796-ae0d-755ad6727655 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:07.464621 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1e766-909d-4334-868c-738753b77444 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:07.472541 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:07.500651 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a5e37-ee65-487c-aaff-fbed71d885b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:07.501807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a5e37-ee65-487c-aaff-fbed71d885b3 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:07.502299 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a5e37-ee65-487c-aaff-fbed71d885b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:07.502814 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a5e37-ee65-487c-aaff-fbed71d885b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:07.502952 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146056eb-9d8f-44bd-b5ef-b83c69994f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:07.503526 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146056eb-9d8f-44bd-b5ef-b83c69994f8b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:07.503810 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146056eb-9d8f-44bd-b5ef-b83c69994f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:07.504612 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146056eb-9d8f-44bd-b5ef-b83c69994f8b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:07.505020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a5e37-ee65-487c-aaff-fbed71d885b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:07.505397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146056eb-9d8f-44bd-b5ef-b83c69994f8b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:07.505776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a5e37-ee65-487c-aaff-fbed71d885b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:07.507076 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146056eb-9d8f-44bd-b5ef-b83c69994f8b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:07.508433 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a5e37-ee65-487c-aaff-fbed71d885b3 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:07.515996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-146056eb-9d8f-44bd-b5ef-b83c69994f8b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:00:07.515996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:07.515996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:00:07.515996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:00:07.515996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:00:07.515996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:00:07.515996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:00:07.515996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:07.515996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:00:07.515996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:07.515996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:07.515996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:07.515996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:07.515996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:07.515996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:07.515996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:07.515996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:07.517029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:07.517029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:07.517029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:07.517029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:07.517029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:07.517029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:07.517029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:00:07.517029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:00:07.517029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:00:07.517029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:00:07.517029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:07.517029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:07.517029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:07.517029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:07.517029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:07.517029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:07.517887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:07.517887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:07.517887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:07.517887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:07.517887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:07.517887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:07.517887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:07.517887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:07.517887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:07.517887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:07.517887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:00:07.517887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:07.517887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:00:07.517887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:07.517887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 353/694] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Oct 17 03:00:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:00:07.525512 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 342/695] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:07.525801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146056eb-9d8f-44bd-b5ef-b83c69994f8b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:07.531266 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a5e37-ee65-487c-aaff-fbed71d885b3 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:07.632877 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146056eb-9d8f-44bd-b5ef-b83c69994f8b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:07.637079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a5e37-ee65-487c-aaff-fbed71d885b3 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:07.644981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:07.670600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c8f29-21bf-4bc6-96ae-91c28ad93a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:07.671190 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c8f29-21bf-4bc6-96ae-91c28ad93a6f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:07.671474 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c8f29-21bf-4bc6-96ae-91c28ad93a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:07.671751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c8f29-21bf-4bc6-96ae-91c28ad93a6f None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:07.672432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c8f29-21bf-4bc6-96ae-91c28ad93a6f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:07.672746 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c8f29-21bf-4bc6-96ae-91c28ad93a6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:07.674333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c8f29-21bf-4bc6-96ae-91c28ad93a6f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:07.677612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917da82c-7e79-4477-87a8-d40cddeef501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:07.678497 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917da82c-7e79-4477-87a8-d40cddeef501 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:07.679040 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917da82c-7e79-4477-87a8-d40cddeef501 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:07.679539 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917da82c-7e79-4477-87a8-d40cddeef501 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:07.683011 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 354/696] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Oct 17 03:00:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:07.685833 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c8f29-21bf-4bc6-96ae-91c28ad93a6f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:07.693580 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917da82c-7e79-4477-87a8-d40cddeef501 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:07.695061 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917da82c-7e79-4477-87a8-d40cddeef501 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:07.707469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-917da82c-7e79-4477-87a8-d40cddeef501 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 03:00:07.707469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:07.707469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:07.707469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:07.707469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:07.707469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:07.707469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:07.707469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:07.707469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:07.707469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:07.707469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:07.707469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:07.707469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:00:07.707469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:00:07.707469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:00:07.707469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:07.708470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:07.709471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:07.709471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:07.709471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:00:07.709471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:00:07.709471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:00:07.709471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:00:07.709471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:00:07.709471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:00:07.709471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 03:00:07.709471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:07.713328 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 343/697] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Oct 17 03:00:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:00:07.714680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917da82c-7e79-4477-87a8-d40cddeef501 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:07.726896 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:07.799368 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c8f29-21bf-4bc6-96ae-91c28ad93a6f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:07.826087 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917da82c-7e79-4477-87a8-d40cddeef501 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:07.835393 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d22c99-c08e-46c1-97a2-13c0d2b8dd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:07.836166 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d22c99-c08e-46c1-97a2-13c0d2b8dd65 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:07.836669 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d22c99-c08e-46c1-97a2-13c0d2b8dd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:07.836999 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d22c99-c08e-46c1-97a2-13c0d2b8dd65 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:07.838023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d22c99-c08e-46c1-97a2-13c0d2b8dd65 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:07.838504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d22c99-c08e-46c1-97a2-13c0d2b8dd65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:07.839965 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d22c99-c08e-46c1-97a2-13c0d2b8dd65 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:07.850449 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 355/698] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Oct 17 03:00:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:07.851502 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d22c99-c08e-46c1-97a2-13c0d2b8dd65 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:07.864083 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e52af-e62c-4651-9cca-6971b067c743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:07.864926 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e52af-e62c-4651-9cca-6971b067c743 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:07.865424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e52af-e62c-4651-9cca-6971b067c743 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:07.865832 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e52af-e62c-4651-9cca-6971b067c743 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:07.866725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e52af-e62c-4651-9cca-6971b067c743 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:07.867215 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e52af-e62c-4651-9cca-6971b067c743 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:07.868890 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e52af-e62c-4651-9cca-6971b067c743 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:07.888087 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 344/699] 10.222.0.27 () {56 vars in 1149 bytes} [Mon Oct 17 03:00:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:07.895081 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e52af-e62c-4651-9cca-6971b067c743 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:07.923480 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:07.953079 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:07.964147 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d22c99-c08e-46c1-97a2-13c0d2b8dd65 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:08.001336 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad075c71-26a5-4006-8351-cdf09b39dba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:08.002055 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad075c71-26a5-4006-8351-cdf09b39dba5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:08.002408 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad075c71-26a5-4006-8351-cdf09b39dba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:08.002754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad075c71-26a5-4006-8351-cdf09b39dba5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:08.003553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad075c71-26a5-4006-8351-cdf09b39dba5 None admin] RBAC: Adding query filter params (interface=public, service_id=f912cb6cfdb14748b81e73efed85ac37, region_id=RegionOne) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:08.004120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad075c71-26a5-4006-8351-cdf09b39dba5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:08.005554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad075c71-26a5-4006-8351-cdf09b39dba5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:08.011438 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 356/700] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Oct 17 03:00:07 2022] GET /identity/v3/endpoints?service_id=f912cb6cfdb14748b81e73efed85ac37&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:08.011906 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e52af-e62c-4651-9cca-6971b067c743 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:08.017139 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad075c71-26a5-4006-8351-cdf09b39dba5 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:08.051941 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db0c4e-96ba-41e3-944e-8a0b9c8c5790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:08.052808 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db0c4e-96ba-41e3-944e-8a0b9c8c5790 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:08.053268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db0c4e-96ba-41e3-944e-8a0b9c8c5790 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:08.053711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db0c4e-96ba-41e3-944e-8a0b9c8c5790 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:08.127758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad075c71-26a5-4006-8351-cdf09b39dba5 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:08.133384 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db0c4e-96ba-41e3-944e-8a0b9c8c5790 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:08.135086 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db0c4e-96ba-41e3-944e-8a0b9c8c5790 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:08.162350 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:08.165490 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a1127-24f9-45b9-aec4-09cd058c8c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:08.166275 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a1127-24f9-45b9-aec4-09cd058c8c02 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:08.166676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a1127-24f9-45b9-aec4-09cd058c8c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:08.167059 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a1127-24f9-45b9-aec4-09cd058c8c02 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:08.167815 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a1127-24f9-45b9-aec4-09cd058c8c02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:08.169653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a1127-24f9-45b9-aec4-09cd058c8c02 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:08.181285 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 357/701] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Oct 17 03:00:08 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:00:08.214073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c6143-c8c5-4b68-b047-5877e840d2d3 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:08.214354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c6143-c8c5-4b68-b047-5877e840d2d3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:08.214625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c6143-c8c5-4b68-b047-5877e840d2d3 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:08.216256 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 358/702] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:00:08.221673 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7643ff5-1ebf-47f6-913e-303bed2062c0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:08.221949 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7643ff5-1ebf-47f6-913e-303bed2062c0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:08.222222 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7643ff5-1ebf-47f6-913e-303bed2062c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:08.241908 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d7643ff5-1ebf-47f6-913e-303bed2062c0 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:08.243168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d7643ff5-1ebf-47f6-913e-303bed2062c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:08.261984 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:08.262237 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 345/703] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 03:00:07 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 03:00:08.264702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db0c4e-96ba-41e3-944e-8a0b9c8c5790 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:08.327587 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 359/704] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:00:08.394571 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db0c4e-96ba-41e3-944e-8a0b9c8c5790 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:08.425487 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:08.437807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98736990-cd5c-46f5-bd94-628911346db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:08.438379 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736990-cd5c-46f5-bd94-628911346db4 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:08.438663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736990-cd5c-46f5-bd94-628911346db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:08.438968 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736990-cd5c-46f5-bd94-628911346db4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:08.450722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98736990-cd5c-46f5-bd94-628911346db4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:08.452251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98736990-cd5c-46f5-bd94-628911346db4 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:08.467725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-98736990-cd5c-46f5-bd94-628911346db4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:08.467725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:08.467725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:08.467725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:08.467725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:08.467725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:08.467725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:08.467725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:08.467725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:08.467725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:08.467725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:08.467725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:08.467725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:00:08.467725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:00:08.467725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:00:08.467725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:00:08.468719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:08.468719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:08.468719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:08.468719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:08.468719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:08.468719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:08.468719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:08.468719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:08.468719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:08.468719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:08.468719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:08.468719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:08.468719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:08.468719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:08.468719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:08.468719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:08.468719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:00:08.470006 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:00:08.470006 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:08.470006 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:08.470006 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 346/705] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:08.475999 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7643ff5-1ebf-47f6-913e-303bed2062c0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:08.483819 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48d400-17c3-4901-9341-0f0dbeb90827 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:08.484097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48d400-17c3-4901-9341-0f0dbeb90827 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:08.484368 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48d400-17c3-4901-9341-0f0dbeb90827 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:08.485946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 347/706] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:00:08.491064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77329b-8f0c-4ef8-b4a5-c89880971f3a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:08.491335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77329b-8f0c-4ef8-b4a5-c89880971f3a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:08.491613 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77329b-8f0c-4ef8-b4a5-c89880971f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:08.510204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-db77329b-8f0c-4ef8-b4a5-c89880971f3a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:08.511408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-db77329b-8f0c-4ef8-b4a5-c89880971f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:08.584469 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7643ff5-1ebf-47f6-913e-303bed2062c0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:08.596511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 348/707] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:00:08.619144 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77329b-8f0c-4ef8-b4a5-c89880971f3a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:08.629779 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b23b38-274b-4849-8d3b-32828bcd0f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:08.630487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b23b38-274b-4849-8d3b-32828bcd0f12 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:08.630844 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b23b38-274b-4849-8d3b-32828bcd0f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:08.631484 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b23b38-274b-4849-8d3b-32828bcd0f12 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:08.632321 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b23b38-274b-4849-8d3b-32828bcd0f12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:08.632697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b23b38-274b-4849-8d3b-32828bcd0f12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:08.634258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b23b38-274b-4849-8d3b-32828bcd0f12 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:08.647817 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 360/708] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:08.653428 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b23b38-274b-4849-8d3b-32828bcd0f12 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:08.667383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:08.747233 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77329b-8f0c-4ef8-b4a5-c89880971f3a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:08.755679 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:08.770782 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b23b38-274b-4849-8d3b-32828bcd0f12 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:08.791597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231eb2bd-515c-408e-a4e7-47488145e087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:08.792381 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231eb2bd-515c-408e-a4e7-47488145e087 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:08.793142 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231eb2bd-515c-408e-a4e7-47488145e087 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:08.793603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231eb2bd-515c-408e-a4e7-47488145e087 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:08.794301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231eb2bd-515c-408e-a4e7-47488145e087 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:08.796527 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231eb2bd-515c-408e-a4e7-47488145e087 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:08.809619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-231eb2bd-515c-408e-a4e7-47488145e087 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:00:08.809619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:08.809619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:00:08.809619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:00:08.809619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:00:08.809619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:00:08.809619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:00:08.809619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:08.809619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:00:08.809619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:08.809619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:08.809619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:08.809619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:08.809619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:08.809619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:08.809619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:08.809619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:08.810917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:08.810917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:08.810917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:08.810917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:08.810917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:08.810917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:08.810917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:00:08.810917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:00:08.810917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:00:08.810917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:00:08.810917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:08.810917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:08.810917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:08.810917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:08.810917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:08.810917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:08.811820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:08.811820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:08.811820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:08.811820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:08.811820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:08.811820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:08.811820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:08.811820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:08.811820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:08.811820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:08.811820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:00:08.811820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:08.811820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:00:08.811820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:08.815695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1149b-5ea1-4e7b-bbfb-f023e078b1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:08.816886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 349/709] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 03:00:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:00:08.817210 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1149b-5ea1-4e7b-bbfb-f023e078b1b6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:08.817705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1149b-5ea1-4e7b-bbfb-f023e078b1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:08.817921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231eb2bd-515c-408e-a4e7-47488145e087 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:08.818324 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1149b-5ea1-4e7b-bbfb-f023e078b1b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:08.827923 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1149b-5ea1-4e7b-bbfb-f023e078b1b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:08.829622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1149b-5ea1-4e7b-bbfb-f023e078b1b6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:08.845733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-77f1149b-5ea1-4e7b-bbfb-f023e078b1b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:00:08.845733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:08.845733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:08.845733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:08.845733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:08.845733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:08.845733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:08.845733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:08.845733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:08.845733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:08.845733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:08.845733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:08.845733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:00:08.845733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:00:08.845733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:00:08.846631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:00:08.846631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:08.846631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:08.846631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:08.846631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:08.846631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:08.846631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:08.846631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:08.846631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:08.846631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:08.846631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:08.846631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:08.846631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:08.846631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:08.846631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:08.846631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:08.846631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:08.847613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:00:08.847613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:00:08.847613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:00:08.847613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:00:08.847613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:00:08.847613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:00:08.847613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:00:08.847613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:08.853305 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 361/710] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 03:00:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:08.855506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1149b-5ea1-4e7b-bbfb-f023e078b1b6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:08.931857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:08.943928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231eb2bd-515c-408e-a4e7-47488145e087 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:08.968973 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1149b-5ea1-4e7b-bbfb-f023e078b1b6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:08.980632 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8a00c-06ed-42f2-b2e5-e07f84012d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:08.981442 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8a00c-06ed-42f2-b2e5-e07f84012d1b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:08.981950 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8a00c-06ed-42f2-b2e5-e07f84012d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:08.982381 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8a00c-06ed-42f2-b2e5-e07f84012d1b None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:08.983270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8a00c-06ed-42f2-b2e5-e07f84012d1b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:08.983714 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8a00c-06ed-42f2-b2e5-e07f84012d1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:08.985363 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8a00c-06ed-42f2-b2e5-e07f84012d1b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:08.991034 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 350/711] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Oct 17 03:00:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:08.992077 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8a00c-06ed-42f2-b2e5-e07f84012d1b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:09.008511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ad7a7-e20a-4876-b507-64293fbc7b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:09.009056 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ad7a7-e20a-4876-b507-64293fbc7b7a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:09.009341 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ad7a7-e20a-4876-b507-64293fbc7b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:09.009617 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ad7a7-e20a-4876-b507-64293fbc7b7a None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:09.010334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ad7a7-e20a-4876-b507-64293fbc7b7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:09.010624 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ad7a7-e20a-4876-b507-64293fbc7b7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:09.012117 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ad7a7-e20a-4876-b507-64293fbc7b7a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:09.026915 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 362/712] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 03:00:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:09.027309 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ad7a7-e20a-4876-b507-64293fbc7b7a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:09.037123 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:09.105124 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8a00c-06ed-42f2-b2e5-e07f84012d1b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:09.126782 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ad7a7-e20a-4876-b507-64293fbc7b7a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:09.139476 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4637117-168f-4f4f-826e-620b7057c470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:09.140111 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4637117-168f-4f4f-826e-620b7057c470 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:09.140392 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4637117-168f-4f4f-826e-620b7057c470 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:09.140674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4637117-168f-4f4f-826e-620b7057c470 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:09.141265 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4637117-168f-4f4f-826e-620b7057c470 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:09.142631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4637117-168f-4f4f-826e-620b7057c470 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:09.152153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-f4637117-168f-4f4f-826e-620b7057c470 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 03:00:09.152153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:09.152153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:00:09.152153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:00:09.152153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:00:09.152153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:00:09.152153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:00:09.152153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:09.152153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 03:00:09.152153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:09.152153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:09.152153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:09.152153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:09.152153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:09.152153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:09.152153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:09.152153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:09.153327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:09.153327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:09.153327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:09.153327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:09.153327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:09.153327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:09.153327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:00:09.153327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:00:09.153327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:00:09.153327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:00:09.153327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:09.153327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:09.153327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:09.153327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:09.153327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:09.153327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:09.154164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:09.154164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:09.154164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:09.154164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:09.154164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:09.154164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:09.154164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:09.154164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:09.154164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:09.154164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:09.154164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:00:09.154164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:09.154164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 03:00:09.154164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:09.154164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 351/713] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Oct 17 03:00:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:09.155163 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4637117-168f-4f4f-826e-620b7057c470 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:09.165449 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:09.172878 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2103da3-ca06-4df1-997d-007183aa4b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:09.173498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2103da3-ca06-4df1-997d-007183aa4b9d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:09.173781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2103da3-ca06-4df1-997d-007183aa4b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:09.174055 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2103da3-ca06-4df1-997d-007183aa4b9d None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:09.174699 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2103da3-ca06-4df1-997d-007183aa4b9d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:09.175032 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2103da3-ca06-4df1-997d-007183aa4b9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:09.176764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2103da3-ca06-4df1-997d-007183aa4b9d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:09.190916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 363/714] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Oct 17 03:00:09 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:09.190916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2103da3-ca06-4df1-997d-007183aa4b9d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:09.303292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:09.310188 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4637117-168f-4f4f-826e-620b7057c470 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:09.321269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2103da3-ca06-4df1-997d-007183aa4b9d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:09.350607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ade0d-ba8f-4b8d-aa29-7f88aa7dca9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:09.351522 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ade0d-ba8f-4b8d-aa29-7f88aa7dca9e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:09.351809 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ade0d-ba8f-4b8d-aa29-7f88aa7dca9e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:09.352348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ade0d-ba8f-4b8d-aa29-7f88aa7dca9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:09.354001 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ade0d-ba8f-4b8d-aa29-7f88aa7dca9e None admin] RBAC: Adding query filter params (role.id=355fbb07456348ac8f6ae42b3dd96690, scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2, user.id=280d47ed79f14105a83b18b2cac58b60) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:09.354465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ade0d-ba8f-4b8d-aa29-7f88aa7dca9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:09.356718 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ade0d-ba8f-4b8d-aa29-7f88aa7dca9e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:09.361540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6bb63-10b6-4067-861b-ab8fd239d757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:09.362168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6bb63-10b6-4067-861b-ab8fd239d757 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:09.362451 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6bb63-10b6-4067-861b-ab8fd239d757 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:09.362728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6bb63-10b6-4067-861b-ab8fd239d757 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:09.363484 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b6bb63-10b6-4067-861b-ab8fd239d757 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:09.363732 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 352/715] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 03:00:09 2022] GET /identity/v3/role_assignments?user.id=280d47ed79f14105a83b18b2cac58b60&scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2&role.id=355fbb07456348ac8f6ae42b3dd96690 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:09.363987 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b6bb63-10b6-4067-861b-ab8fd239d757 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:09.365623 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b6bb63-10b6-4067-861b-ab8fd239d757 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:09.371723 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 364/716] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Oct 17 03:00:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:09.380475 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ade0d-ba8f-4b8d-aa29-7f88aa7dca9e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:09.501759 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ade0d-ba8f-4b8d-aa29-7f88aa7dca9e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:09.540097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322605e-ae93-446a-ab97-d5a7f63cf569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:09.540097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322605e-ae93-446a-ab97-d5a7f63cf569 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:09.540352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322605e-ae93-446a-ab97-d5a7f63cf569 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:09.540602 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322605e-ae93-446a-ab97-d5a7f63cf569 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:09.541205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322605e-ae93-446a-ab97-d5a7f63cf569 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:09.541503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322605e-ae93-446a-ab97-d5a7f63cf569 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:09.542897 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322605e-ae93-446a-ab97-d5a7f63cf569 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:09.549230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 353/717] 10.222.0.27 () {56 vars in 1114 bytes} [Mon Oct 17 03:00:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:09.555440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6bb63-10b6-4067-861b-ab8fd239d757 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:09.620920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:09.663832 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6bb63-10b6-4067-861b-ab8fd239d757 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:09.697777 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7dc144-96f7-4a8f-8784-ae50abe00e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:09.698386 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dc144-96f7-4a8f-8784-ae50abe00e91 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:09.698663 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dc144-96f7-4a8f-8784-ae50abe00e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:09.698961 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dc144-96f7-4a8f-8784-ae50abe00e91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:09.699637 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7dc144-96f7-4a8f-8784-ae50abe00e91 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:09.701048 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7dc144-96f7-4a8f-8784-ae50abe00e91 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:09.725409 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 365/718] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:00:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:00:10.532513 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c92260-a968-478c-87f3-c071471c085e None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:10.533008 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c92260-a968-478c-87f3-c071471c085e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:10.533008 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c92260-a968-478c-87f3-c071471c085e None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:10.534296 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 354/719] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:10.540533 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc511cd6-3d07-4c9c-9216-7f511c2b40a3 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:10.540974 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc511cd6-3d07-4c9c-9216-7f511c2b40a3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:10.541155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc511cd6-3d07-4c9c-9216-7f511c2b40a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:10.560709 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-dc511cd6-3d07-4c9c-9216-7f511c2b40a3 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:10.562182 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dc511cd6-3d07-4c9c-9216-7f511c2b40a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:10.639025 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 366/720] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:10.908239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c92260-a968-478c-87f3-c071471c085e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:10.936867 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:11.017525 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c92260-a968-478c-87f3-c071471c085e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:11.054980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5c5ad-3174-4ddb-90c7-701bfb7536dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:11.055619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5c5ad-3174-4ddb-90c7-701bfb7536dc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:11.055901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5c5ad-3174-4ddb-90c7-701bfb7536dc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:11.056176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5c5ad-3174-4ddb-90c7-701bfb7536dc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:11.056767 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5c5ad-3174-4ddb-90c7-701bfb7536dc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:11.058150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5c5ad-3174-4ddb-90c7-701bfb7536dc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:11.067554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-4fe5c5ad-3174-4ddb-90c7-701bfb7536dc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:00:11.067554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:11.067554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:00:11.067554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:00:11.067554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:00:11.067554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:00:11.067554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:00:11.067554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:11.067554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:00:11.067554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:11.067554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:11.067554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:11.067554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:11.067554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:11.067554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:11.067554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:11.067554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:11.068543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:11.068543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:11.068543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:11.068543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:11.068543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:11.068543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:11.068543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:00:11.068543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:00:11.068543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:00:11.068543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:00:11.068543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:11.068543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:11.068543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:11.068543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:11.068543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:11.068543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:11.069970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:11.069970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:11.069970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:11.069970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:11.069970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:11.069970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:11.069970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:11.069970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:11.069970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:11.069970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:11.069970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:00:11.069970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:11.069970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:00:11.069970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:11.069970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 355/721] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Oct 17 03:00:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:11.074558 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc511cd6-3d07-4c9c-9216-7f511c2b40a3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:11.119246 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:11.177203 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc511cd6-3d07-4c9c-9216-7f511c2b40a3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:11.212120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d10e7d-14be-4279-961c-e605d439b4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:11.212708 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d10e7d-14be-4279-961c-e605d439b4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:11.212987 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d10e7d-14be-4279-961c-e605d439b4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:11.213255 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d10e7d-14be-4279-961c-e605d439b4d5 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:11.213884 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d10e7d-14be-4279-961c-e605d439b4d5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:11.214185 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d10e7d-14be-4279-961c-e605d439b4d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:11.215570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d10e7d-14be-4279-961c-e605d439b4d5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:11.220843 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 367/722] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Oct 17 03:00:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:11.226777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5c5ad-3174-4ddb-90c7-701bfb7536dc None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:11.335248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:11.339386 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5c5ad-3174-4ddb-90c7-701bfb7536dc None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:11.377780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2574c-1639-4c06-a3c5-ff8ce188e6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:11.379425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2574c-1639-4c06-a3c5-ff8ce188e6ca None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:11.379996 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2574c-1639-4c06-a3c5-ff8ce188e6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:11.380626 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2574c-1639-4c06-a3c5-ff8ce188e6ca None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:11.382162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2574c-1639-4c06-a3c5-ff8ce188e6ca None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:11.382732 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2574c-1639-4c06-a3c5-ff8ce188e6ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:11.384403 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2574c-1639-4c06-a3c5-ff8ce188e6ca None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:11.390847 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 356/723] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Oct 17 03:00:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:11.402920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d10e7d-14be-4279-961c-e605d439b4d5 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:11.493863 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc99f4-bf6e-46fa-91d9-f50b4f6a4423 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:11.494068 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc99f4-bf6e-46fa-91d9-f50b4f6a4423 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:11.494332 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc99f4-bf6e-46fa-91d9-f50b4f6a4423 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:11.496190 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 357/724] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:11.502425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e381ab0-8b69-49f7-bd2e-be2aa79c815a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:11.502806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e381ab0-8b69-49f7-bd2e-be2aa79c815a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:11.503038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e381ab0-8b69-49f7-bd2e-be2aa79c815a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:11.505303 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d10e7d-14be-4279-961c-e605d439b4d5 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:11.509108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:11.524257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-6e381ab0-8b69-49f7-bd2e-be2aa79c815a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:11.525492 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6e381ab0-8b69-49f7-bd2e-be2aa79c815a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:11.545252 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274aafa4-9421-4370-96b7-b851e452cfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:11.546371 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274aafa4-9421-4370-96b7-b851e452cfaf None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:11.546757 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274aafa4-9421-4370-96b7-b851e452cfaf None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:11.547181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274aafa4-9421-4370-96b7-b851e452cfaf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:11.547983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274aafa4-9421-4370-96b7-b851e452cfaf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:11.550402 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274aafa4-9421-4370-96b7-b851e452cfaf None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:11.578985 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 368/725] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:00:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:00:11.619093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 358/726] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:11.687729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b71ab-1ab7-4f40-b4ac-1779f16b3169 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:11.688056 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b71ab-1ab7-4f40-b4ac-1779f16b3169 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:11.688293 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b71ab-1ab7-4f40-b4ac-1779f16b3169 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:11.689912 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 369/727] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:11.696462 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943235c-d5f8-4128-93b6-cccefa308376 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:11.696760 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943235c-d5f8-4128-93b6-cccefa308376 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:11.697018 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943235c-d5f8-4128-93b6-cccefa308376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:11.721530 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4943235c-d5f8-4128-93b6-cccefa308376 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:11.722831 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4943235c-d5f8-4128-93b6-cccefa308376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:11.797784 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 359/728] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:11.896862 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b71ab-1ab7-4f40-b4ac-1779f16b3169 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:11.982933 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816eeb9-4636-4b1e-9500-06efe35c852e None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:11.983225 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816eeb9-4636-4b1e-9500-06efe35c852e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:11.983486 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816eeb9-4636-4b1e-9500-06efe35c852e None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:11.985044 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 360/729] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:11.990625 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49023f-3383-43c7-ae70-ea63c9d9b702 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:11.990932 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49023f-3383-43c7-ae70-ea63c9d9b702 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:11.991215 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49023f-3383-43c7-ae70-ea63c9d9b702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:12.011278 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8b49023f-3383-43c7-ae70-ea63c9d9b702 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:12.012665 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8b49023f-3383-43c7-ae70-ea63c9d9b702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:12.013148 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b71ab-1ab7-4f40-b4ac-1779f16b3169 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:12.017530 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:12.059319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579376f-b302-4545-b84a-98e57aec7f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:12.060653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579376f-b302-4545-b84a-98e57aec7f08 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:12.060955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579376f-b302-4545-b84a-98e57aec7f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:12.061237 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579376f-b302-4545-b84a-98e57aec7f08 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:12.061975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579376f-b302-4545-b84a-98e57aec7f08 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:12.063440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579376f-b302-4545-b84a-98e57aec7f08 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:12.083083 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 370/730] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Oct 17 03:00:11 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:00:12.083939 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579376f-b302-4545-b84a-98e57aec7f08 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:12.111189 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 361/731] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Oct 17 03:00:12.123500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:12.224143 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579376f-b302-4545-b84a-98e57aec7f08 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:12.266027 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131d0ce-a404-4585-a75b-f9a13ad5df43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:12.266671 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131d0ce-a404-4585-a75b-f9a13ad5df43 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:12.266979 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131d0ce-a404-4585-a75b-f9a13ad5df43 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:12.267263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131d0ce-a404-4585-a75b-f9a13ad5df43 None admin] PATH_INFO: `/v3/services/image` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:12.267858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131d0ce-a404-4585-a75b-f9a13ad5df43 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:12.269226 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131d0ce-a404-4585-a75b-f9a13ad5df43 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:12.283470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-a131d0ce-a404-4585-a75b-f9a13ad5df43 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:00:12.283470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:12.283470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:00:12.283470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:00:12.283470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:00:12.283470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:00:12.283470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:00:12.283470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:12.283470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:00:12.283470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:12.283470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:12.283470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:12.283470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:12.283470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:12.283470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:12.283470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:12.283470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:12.284496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:12.284496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:12.284496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:12.284496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:12.284496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:12.284496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:12.284496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:00:12.284496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:00:12.284496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:00:12.284496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:00:12.284496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:12.284496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:12.284496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:12.284496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:12.284496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:12.284496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:12.285954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:12.285954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:12.285954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:12.285954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:12.285954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:12.285954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:12.285954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:12.285954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:12.285954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:12.285954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:12.285954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:00:12.285954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:12.285954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:00:12.285954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:12.287423 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 371/732] 10.222.0.27 () {56 vars in 1105 bytes} [Mon Oct 17 03:00:12 2022] GET /identity/v3/services/image => generated 86 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:12.291406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49023f-3383-43c7-ae70-ea63c9d9b702 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:12.395121 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131d0ce-a404-4585-a75b-f9a13ad5df43 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:12.402050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b49023f-3383-43c7-ae70-ea63c9d9b702 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:12.444218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9373d-a7d9-4415-a673-7632399439f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:12.445243 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9373d-a7d9-4415-a673-7632399439f2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:12.445719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9373d-a7d9-4415-a673-7632399439f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:12.446186 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9373d-a7d9-4415-a673-7632399439f2 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:12.447137 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9373d-a7d9-4415-a673-7632399439f2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:12.447731 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9373d-a7d9-4415-a673-7632399439f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:12.449309 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9373d-a7d9-4415-a673-7632399439f2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:12.451583 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:12.456192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 362/733] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:12.462647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9373d-a7d9-4415-a673-7632399439f2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:12.502632 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:12.524431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131d0ce-a404-4585-a75b-f9a13ad5df43 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:12.575144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a336d95-f636-4eda-8d12-640c149b7756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:12.575778 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a336d95-f636-4eda-8d12-640c149b7756 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:12.576037 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a336d95-f636-4eda-8d12-640c149b7756 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:12.576315 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a336d95-f636-4eda-8d12-640c149b7756 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:12.577430 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a336d95-f636-4eda-8d12-640c149b7756 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:12.579190 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a336d95-f636-4eda-8d12-640c149b7756 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:12.583364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: WARNING keystone.server.flask.common [None req-9a336d95-f636-4eda-8d12-640c149b7756 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 17 03:00:12.584874 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9373d-a7d9-4415-a673-7632399439f2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:12.608173 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.sql.core [None req-9a336d95-f636-4eda-8d12-640c149b7756 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 03:00:12.608173 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 03:00:12.608173 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [parameters: {'id': 'ed5e527892e7446786b1273915b886fb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 17 03:00:12.608173 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81582) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 03:00:12.618665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-9a336d95-f636-4eda-8d12-640c149b7756 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ed5e527892e7446786b1273915b886fb.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ed5e527892e7446786b1273915b886fb. Oct 17 03:00:12.618665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:12.618665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 03:00:12.618665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 03:00:12.618665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 03:00:12.618665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 03:00:12.618665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 03:00:12.618665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 03:00:12.618665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 03:00:12.618665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application conn.query(q) Oct 17 03:00:12.618665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 03:00:12.618665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 03:00:12.618665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result.read() Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 03:00:12.619600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 03:00:12.620579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 03:00:12.620579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application next(self.gen) Oct 17 03:00:12.620579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 03:00:12.620579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application yield resource Oct 17 03:00:12.620579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 03:00:12.620579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application next(self.gen) Oct 17 03:00:12.620579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 03:00:12.620579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 03:00:12.620579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 03:00:12.620579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 03:00:12.620579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 03:00:12.620579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise self.value Oct 17 03:00:12.620579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 03:00:12.620579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 03:00:12.620579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 03:00:12.620579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application session.commit() Oct 17 03:00:12.620579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 03:00:12.621507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 03:00:12.621507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 03:00:12.621507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 03:00:12.621507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 03:00:12.621507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.session.flush() Oct 17 03:00:12.621507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 03:00:12.621507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 03:00:12.621507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 03:00:12.621507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 03:00:12.621507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 03:00:12.621507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application compat.raise_( Oct 17 03:00:12.621507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 03:00:12.621507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception Oct 17 03:00:12.621507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 03:00:12.621507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 03:00:12.621507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 03:00:12.621507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 03:00:12.623155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 03:00:12.623155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 03:00:12.623155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 03:00:12.623155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 03:00:12.623155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 03:00:12.623155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 03:00:12.623155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 03:00:12.623155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 03:00:12.623155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 03:00:12.623155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 03:00:12.623155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 03:00:12.623155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 03:00:12.623155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 03:00:12.623155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 03:00:12.623155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 03:00:12.623983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 03:00:12.623983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 03:00:12.623983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception Oct 17 03:00:12.623983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 03:00:12.623983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 03:00:12.623983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 03:00:12.623983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 03:00:12.623983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 03:00:12.623983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 03:00:12.623983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 03:00:12.623983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application conn.query(q) Oct 17 03:00:12.623983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 03:00:12.623983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 03:00:12.623983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 03:00:12.623983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result.read() Oct 17 03:00:12.623983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 03:00:12.623983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 03:00:12.626253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 03:00:12.626253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 03:00:12.626253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 03:00:12.626253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 03:00:12.626253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 03:00:12.626253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 03:00:12.626253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 03:00:12.626253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 03:00:12.626253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [parameters: {'id': 'ed5e527892e7446786b1273915b886fb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 17 03:00:12.626253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 03:00:12.626253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:12.626253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:12.626253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:12.626253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:12.626253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 03:00:12.628458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 03:00:12.628458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 03:00:12.628458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 03:00:12.628458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 17 03:00:12.628458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:12.628458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:12.628458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:12.628458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:12.628458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:12.628458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:12.628458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:12.628458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:12.628458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:12.628458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:12.628458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:12.628458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:12.628458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:12.631539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:12.631539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 17 03:00:12.631539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 17 03:00:12.631539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:12.631539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:12.631539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 03:00:12.631539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 03:00:12.631539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ed5e527892e7446786b1273915b886fb. Oct 17 03:00:12.631539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:12.631539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 372/734] 10.222.0.27 () {60 vars in 1140 bytes} [Mon Oct 17 03:00:12 2022] POST /identity/v3/projects => generated 262 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:00:12.631539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a336d95-f636-4eda-8d12-640c149b7756 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:12.636975 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238de3f-d24c-4782-bc57-747217a32f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:12.638270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238de3f-d24c-4782-bc57-747217a32f12 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:12.638892 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238de3f-d24c-4782-bc57-747217a32f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:12.639438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238de3f-d24c-4782-bc57-747217a32f12 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:12.640485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238de3f-d24c-4782-bc57-747217a32f12 None admin] RBAC: Adding query filter params (type=image) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:12.640997 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238de3f-d24c-4782-bc57-747217a32f12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:12.643119 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238de3f-d24c-4782-bc57-747217a32f12 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:12.649611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 363/735] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:12 2022] GET /identity/v3/services?type=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:12.764420 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a336d95-f636-4eda-8d12-640c149b7756 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:12.804636 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537d763-72bd-468d-9443-2de982e51631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:12.805649 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537d763-72bd-468d-9443-2de982e51631 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:12.806141 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537d763-72bd-468d-9443-2de982e51631 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:12.806621 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537d763-72bd-468d-9443-2de982e51631 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:12.814767 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537d763-72bd-468d-9443-2de982e51631 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:12.817365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537d763-72bd-468d-9443-2de982e51631 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:12.824216 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:12.834705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-8537d763-72bd-468d-9443-2de982e51631 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:00:12.834705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:12.834705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:12.834705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:12.834705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:12.834705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:12.834705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:12.834705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:12.834705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:12.834705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:12.834705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:12.834705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:12.834705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:00:12.834705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:00:12.834705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:00:12.835657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:00:12.835657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:12.835657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:12.835657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:12.835657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:12.835657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:12.835657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:12.835657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:12.835657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:12.835657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:12.835657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:12.835657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:12.835657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:12.835657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:12.835657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:12.835657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:12.835657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:12.836555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:00:12.836555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:00:12.836555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:00:12.836555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:00:12.836555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:00:12.836555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:00:12.836555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:00:12.836555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:12.838706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 373/736] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 03:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:12.844303 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238de3f-d24c-4782-bc57-747217a32f12 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:12.915718 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:12.961764 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238de3f-d24c-4782-bc57-747217a32f12 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:12.999031 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8c231-3eb1-48f9-9f0f-d9116b844649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:13.000017 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8c231-3eb1-48f9-9f0f-d9116b844649 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:13.000514 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8c231-3eb1-48f9-9f0f-d9116b844649 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:13.000992 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8c231-3eb1-48f9-9f0f-d9116b844649 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:13.001875 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8c231-3eb1-48f9-9f0f-d9116b844649 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:13.002405 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8c231-3eb1-48f9-9f0f-d9116b844649 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:13.004118 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8c231-3eb1-48f9-9f0f-d9116b844649 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:13.023574 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 364/737] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Oct 17 03:00:12 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:13.908292 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b63cd3-a03a-4912-9adb-231788e2fa5b None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:13.908842 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b63cd3-a03a-4912-9adb-231788e2fa5b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:13.908842 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b63cd3-a03a-4912-9adb-231788e2fa5b None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:13.913272 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 374/738] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:13.918839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4bab8-4334-4d52-ba88-1d29cf1d74af None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:13.919285 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4bab8-4334-4d52-ba88-1d29cf1d74af None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:13.919466 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4bab8-4334-4d52-ba88-1d29cf1d74af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:13.938288 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-36c4bab8-4334-4d52-ba88-1d29cf1d74af None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:13.939501 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-36c4bab8-4334-4d52-ba88-1d29cf1d74af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:14.013481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 365/739] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:14.284578 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b63cd3-a03a-4912-9adb-231788e2fa5b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:14.401172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b63cd3-a03a-4912-9adb-231788e2fa5b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:14.412872 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d700ce-ce8d-4e6c-832c-239f1c02c12a None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:14.413480 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d700ce-ce8d-4e6c-832c-239f1c02c12a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:14.413961 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d700ce-ce8d-4e6c-832c-239f1c02c12a None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:14.419005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 366/740] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:00:14.421716 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78e5e4-3af5-4bec-89ca-abf31e94f374 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:14.422242 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78e5e4-3af5-4bec-89ca-abf31e94f374 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:14.423204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f78e5e4-3af5-4bec-89ca-abf31e94f374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:14.439982 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771396a8-9fe9-4e88-beec-abd3f4ece2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:14.440617 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771396a8-9fe9-4e88-beec-abd3f4ece2bf None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:14.440894 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771396a8-9fe9-4e88-beec-abd3f4ece2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:14.441162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771396a8-9fe9-4e88-beec-abd3f4ece2bf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:14.442318 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771396a8-9fe9-4e88-beec-abd3f4ece2bf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:14.443721 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771396a8-9fe9-4e88-beec-abd3f4ece2bf None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:14.448719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4f78e5e4-3af5-4bec-89ca-abf31e94f374 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:14.450229 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4f78e5e4-3af5-4bec-89ca-abf31e94f374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:14.452553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-771396a8-9fe9-4e88-beec-abd3f4ece2bf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:00:14.452553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:14.452553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:00:14.452553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:00:14.452553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:00:14.452553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:00:14.452553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:00:14.452553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:14.452553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:00:14.452553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:14.452553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:14.452553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:14.452553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:14.452553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:14.452553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:14.452553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:14.452553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:14.453552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:14.453552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:14.453552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:14.453552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:14.453552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:14.453552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:14.453552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:00:14.453552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:00:14.453552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:00:14.453552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:00:14.453552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:14.453552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:14.453552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:14.453552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:14.453552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:14.453552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:14.454991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:14.454991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:14.454991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:14.454991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:14.454991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:14.454991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:14.454991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:14.454991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:14.454991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:14.454991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:14.454991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:00:14.454991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:14.454991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:00:14.454991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:14.454991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 375/741] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Oct 17 03:00:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:14.459825 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771396a8-9fe9-4e88-beec-abd3f4ece2bf None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:14.526139 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:14.530235 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:14.555302 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 367/742] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:14.575197 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771396a8-9fe9-4e88-beec-abd3f4ece2bf None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:14.612589 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a829dc2-a8d9-4be9-8772-f87cd19bc5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:14.613204 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a829dc2-a8d9-4be9-8772-f87cd19bc5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:14.613752 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a829dc2-a8d9-4be9-8772-f87cd19bc5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:14.614034 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a829dc2-a8d9-4be9-8772-f87cd19bc5e7 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:14.614670 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a829dc2-a8d9-4be9-8772-f87cd19bc5e7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:14.615066 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a829dc2-a8d9-4be9-8772-f87cd19bc5e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:14.616440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a829dc2-a8d9-4be9-8772-f87cd19bc5e7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:14.621935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 376/743] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Oct 17 03:00:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:14.627991 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78e5e4-3af5-4bec-89ca-abf31e94f374 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:14.733255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f78e5e4-3af5-4bec-89ca-abf31e94f374 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:14.770330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005b2b3-f6c3-449b-96dc-c34942722307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:14.771437 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b2b3-f6c3-449b-96dc-c34942722307 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:14.771927 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b2b3-f6c3-449b-96dc-c34942722307 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:14.772402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b2b3-f6c3-449b-96dc-c34942722307 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:14.773251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005b2b3-f6c3-449b-96dc-c34942722307 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:14.773768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005b2b3-f6c3-449b-96dc-c34942722307 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:14.775367 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005b2b3-f6c3-449b-96dc-c34942722307 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:14.780876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 368/744] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Oct 17 03:00:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:14.789183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a829dc2-a8d9-4be9-8772-f87cd19bc5e7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:14.857544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005b2b3-f6c3-449b-96dc-c34942722307 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:14.905963 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a829dc2-a8d9-4be9-8772-f87cd19bc5e7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:14.923725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:14.934419 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:14.944200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315ac2d-95df-4257-ad29-ae3469b3a50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:14.944816 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315ac2d-95df-4257-ad29-ae3469b3a50f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:14.945094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315ac2d-95df-4257-ad29-ae3469b3a50f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:14.945363 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315ac2d-95df-4257-ad29-ae3469b3a50f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:14.946071 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315ac2d-95df-4257-ad29-ae3469b3a50f None admin] RBAC: Adding query filter params (interface=admin, service_id=f912cb6cfdb14748b81e73efed85ac37, region_id=RegionOne) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:14.946382 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315ac2d-95df-4257-ad29-ae3469b3a50f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:14.947752 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315ac2d-95df-4257-ad29-ae3469b3a50f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:14.953295 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 377/745] 10.222.0.27 () {56 vars in 1249 bytes} [Mon Oct 17 03:00:14 2022] GET /identity/v3/endpoints?service_id=f912cb6cfdb14748b81e73efed85ac37&interface=admin®ion_id=RegionOne => generated 193 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:14.954527 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50c190-d5db-47b7-9d72-4dfde0b1d173 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:14.954811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50c190-d5db-47b7-9d72-4dfde0b1d173 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:14.955129 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50c190-d5db-47b7-9d72-4dfde0b1d173 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:14.958477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 378/746] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:14.959366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50c190-d5db-47b7-9d72-4dfde0b1d173 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:14.985233 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005b2b3-f6c3-449b-96dc-c34942722307 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:15.021795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4a75a-6997-4687-b84c-e66dd6445de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:15.022429 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4a75a-6997-4687-b84c-e66dd6445de9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:15.022707 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4a75a-6997-4687-b84c-e66dd6445de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:15.023010 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4a75a-6997-4687-b84c-e66dd6445de9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:15.023595 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4a75a-6997-4687-b84c-e66dd6445de9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:15.024950 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e4a75a-6997-4687-b84c-e66dd6445de9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:15.033561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-86e4a75a-6997-4687-b84c-e66dd6445de9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:00:15.033561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:15.033561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:00:15.033561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:00:15.033561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:00:15.033561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:00:15.033561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:00:15.033561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:15.033561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:00:15.033561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:15.033561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:15.033561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:15.033561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:15.033561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:15.033561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:15.033561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:15.033561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:15.034741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:15.034741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:15.034741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:15.034741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:15.034741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:15.034741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:15.034741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:00:15.034741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:00:15.034741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:00:15.034741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:00:15.034741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:15.034741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:15.034741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:15.034741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:15.034741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:15.034741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:15.036372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:15.036372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:15.036372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:15.036372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:15.036372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:15.036372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:15.036372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:15.036372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:15.036372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:15.036372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:15.036372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:00:15.036372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:15.036372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:00:15.036372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:15.036372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 369/747] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 03:00:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:00:15.037277 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4003196-05b6-456a-85d3-08fe0cc96bf6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:15.037277 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4003196-05b6-456a-85d3-08fe0cc96bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:15.037277 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4003196-05b6-456a-85d3-08fe0cc96bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:15.056357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f4003196-05b6-456a-85d3-08fe0cc96bf6 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:15.057800 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f4003196-05b6-456a-85d3-08fe0cc96bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:15.079448 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50c190-d5db-47b7-9d72-4dfde0b1d173 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:15.116115 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568adcaf-b2cd-4c73-9eca-89abe3ae1d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:15.116727 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568adcaf-b2cd-4c73-9eca-89abe3ae1d79 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:15.117012 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568adcaf-b2cd-4c73-9eca-89abe3ae1d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:15.117288 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568adcaf-b2cd-4c73-9eca-89abe3ae1d79 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:15.117892 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568adcaf-b2cd-4c73-9eca-89abe3ae1d79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:15.119291 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568adcaf-b2cd-4c73-9eca-89abe3ae1d79 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:15.125617 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 379/748] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Oct 17 03:00:14 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:00:15.126384 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568adcaf-b2cd-4c73-9eca-89abe3ae1d79 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:15.138033 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 370/749] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:15.196208 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:15.252153 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568adcaf-b2cd-4c73-9eca-89abe3ae1d79 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:15.291129 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35c1b5-9c66-434d-ac8a-8f9de937e592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:15.291682 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35c1b5-9c66-434d-ac8a-8f9de937e592 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:15.291962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35c1b5-9c66-434d-ac8a-8f9de937e592 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:15.292233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35c1b5-9c66-434d-ac8a-8f9de937e592 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:15.292881 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35c1b5-9c66-434d-ac8a-8f9de937e592 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:15.293184 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35c1b5-9c66-434d-ac8a-8f9de937e592 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:15.294727 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35c1b5-9c66-434d-ac8a-8f9de937e592 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:15.301313 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 380/750] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Oct 17 03:00:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:15.306774 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4003196-05b6-456a-85d3-08fe0cc96bf6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:15.357927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2549a-de5e-4a21-bff5-e54708e95df5 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:15.358248 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2549a-de5e-4a21-bff5-e54708e95df5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:15.358474 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2549a-de5e-4a21-bff5-e54708e95df5 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:15.359960 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 381/751] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:00:15.362195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:15.366018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8ed4c-8ea5-4951-84ca-ab955f8ab9fb None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:15.366470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8ed4c-8ea5-4951-84ca-ab955f8ab9fb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:15.366906 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8ed4c-8ea5-4951-84ca-ab955f8ab9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:15.386308 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-51b8ed4c-8ea5-4951-84ca-ab955f8ab9fb None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:15.388212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-51b8ed4c-8ea5-4951-84ca-ab955f8ab9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:15.429050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4003196-05b6-456a-85d3-08fe0cc96bf6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:15.459500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 382/752] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:00:15.460612 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8ed4c-8ea5-4951-84ca-ab955f8ab9fb None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:15.469215 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090afd7-0e76-4ab6-9ed3-17385659156f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:15.469830 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090afd7-0e76-4ab6-9ed3-17385659156f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:15.470095 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090afd7-0e76-4ab6-9ed3-17385659156f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:15.470368 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090afd7-0e76-4ab6-9ed3-17385659156f None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:15.471029 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090afd7-0e76-4ab6-9ed3-17385659156f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:15.471330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090afd7-0e76-4ab6-9ed3-17385659156f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:15.472853 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090afd7-0e76-4ab6-9ed3-17385659156f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:15.482497 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 371/753] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Oct 17 03:00:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:15.484131 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090afd7-0e76-4ab6-9ed3-17385659156f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:15.525378 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:15.575336 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8ed4c-8ea5-4951-84ca-ab955f8ab9fb None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:15.595870 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090afd7-0e76-4ab6-9ed3-17385659156f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:15.604419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:15.610817 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9177d4-24e4-41da-afc5-1ecb616aae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:15.611487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9177d4-24e4-41da-afc5-1ecb616aae80 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:15.611767 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9177d4-24e4-41da-afc5-1ecb616aae80 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:15.612119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9177d4-24e4-41da-afc5-1ecb616aae80 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:15.612741 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9177d4-24e4-41da-afc5-1ecb616aae80 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:15.614428 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9177d4-24e4-41da-afc5-1ecb616aae80 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:15.636412 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 383/754] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Oct 17 03:00:15 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:00:15.656181 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d99b27-d094-4a90-896b-364113f332e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:15.657225 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d99b27-d094-4a90-896b-364113f332e6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:15.657769 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d99b27-d094-4a90-896b-364113f332e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:15.658269 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d99b27-d094-4a90-896b-364113f332e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:15.659338 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d99b27-d094-4a90-896b-364113f332e6 None admin] RBAC: Adding query filter params (interface=public, service_id=842869a52f1f42cb83cba09e3554563d, region_id=RegionOne) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:15.659873 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d99b27-d094-4a90-896b-364113f332e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:15.661791 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d99b27-d094-4a90-896b-364113f332e6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:15.668829 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 372/755] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Oct 17 03:00:15 2022] GET /identity/v3/endpoints?service_id=842869a52f1f42cb83cba09e3554563d&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:15.675067 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9177d4-24e4-41da-afc5-1ecb616aae80 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:15.740791 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d99b27-d094-4a90-896b-364113f332e6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:15.810086 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9177d4-24e4-41da-afc5-1ecb616aae80 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:15.824341 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:15.856184 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3f533-2535-45ef-851a-8239b25b59a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:15.856184 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3f533-2535-45ef-851a-8239b25b59a6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:15.856184 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3f533-2535-45ef-851a-8239b25b59a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:15.856523 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3f533-2535-45ef-851a-8239b25b59a6 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:15.857244 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3f533-2535-45ef-851a-8239b25b59a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:15.858631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3f533-2535-45ef-851a-8239b25b59a6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:15.867742 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 384/756] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Oct 17 03:00:15 2022] GET /identity/v3/services => generated 1957 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:00:15.874243 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d99b27-d094-4a90-896b-364113f332e6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:15.918697 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cc965-436a-437e-b86d-8ec46b33aed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:15.919435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cc965-436a-437e-b86d-8ec46b33aed7 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:15.919717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cc965-436a-437e-b86d-8ec46b33aed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:15.919993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cc965-436a-437e-b86d-8ec46b33aed7 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:15.920656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9cc965-436a-437e-b86d-8ec46b33aed7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:15.922134 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9cc965-436a-437e-b86d-8ec46b33aed7 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:15.924875 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: WARNING keystone.server.flask.common [None req-ab9cc965-436a-437e-b86d-8ec46b33aed7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 17 03:00:15.940604 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.sql.core [None req-ab9cc965-436a-437e-b86d-8ec46b33aed7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 03:00:15.940604 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 17 03:00:15.940604 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [parameters: {'user_id': '01c9b08cfe0f41b087caae7cf57bdf05', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 17 03:00:15.940604 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81581) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 03:00:15.959981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-ab9cc965-436a-437e-b86d-8ec46b33aed7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 17 03:00:15.959981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:15.959981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 03:00:15.959981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 03:00:15.959981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 03:00:15.959981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 03:00:15.959981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 03:00:15.959981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 03:00:15.959981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 03:00:15.959981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application conn.query(q) Oct 17 03:00:15.959981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 03:00:15.959981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 03:00:15.959981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 03:00:15.959981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result.read() Oct 17 03:00:15.959981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 03:00:15.960989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application next(self.gen) Oct 17 03:00:15.961995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 03:00:15.961995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application yield resource Oct 17 03:00:15.961995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 03:00:15.961995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application next(self.gen) Oct 17 03:00:15.961995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 03:00:15.961995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 03:00:15.961995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 03:00:15.961995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 03:00:15.961995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 03:00:15.961995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise self.value Oct 17 03:00:15.961995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 03:00:15.961995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 03:00:15.961995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 03:00:15.961995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application session.commit() Oct 17 03:00:15.961995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 03:00:15.961995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 03:00:15.961995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 03:00:15.963075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 03:00:15.963075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 03:00:15.963075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.session.flush() Oct 17 03:00:15.963075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 03:00:15.963075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 03:00:15.963075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 03:00:15.963075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 03:00:15.963075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 03:00:15.963075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application compat.raise_( Oct 17 03:00:15.963075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 03:00:15.963075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception Oct 17 03:00:15.963075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 03:00:15.963075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 03:00:15.963075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 03:00:15.963075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 03:00:15.963075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 03:00:15.963075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 03:00:15.964214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 03:00:15.964214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 03:00:15.964214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 17 03:00:15.964214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 17 03:00:15.964214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 03:00:15.964214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 03:00:15.964214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 03:00:15.964214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 03:00:15.964214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 03:00:15.964214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 03:00:15.964214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 03:00:15.964214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 03:00:15.964214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 03:00:15.964214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 03:00:15.964214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 03:00:15.964214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception Oct 17 03:00:15.965310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 03:00:15.965310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 03:00:15.965310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 03:00:15.965310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 03:00:15.965310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 03:00:15.965310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 03:00:15.965310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 03:00:15.965310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application conn.query(q) Oct 17 03:00:15.965310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 03:00:15.965310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 03:00:15.965310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 03:00:15.965310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application result.read() Oct 17 03:00:15.965310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 03:00:15.965310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 03:00:15.965310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 03:00:15.965310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 03:00:15.966329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 03:00:15.966329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 03:00:15.966329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 03:00:15.966329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 03:00:15.966329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 03:00:15.966329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 17 03:00:15.966329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [parameters: {'user_id': '01c9b08cfe0f41b087caae7cf57bdf05', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 17 03:00:15.966329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 03:00:15.966329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:15.966329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:15.966329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:15.966329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:15.966329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:15.966329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:15.966329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:15.967465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:15.967465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:15.967465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:15.967465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:15.967465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:15.967465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:15.967465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:15.967465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 17 03:00:15.967465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 17 03:00:15.967465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:15.967465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:15.967465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:00:15.967465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:00:15.967465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:00:15.967465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:00:15.967465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 17 03:00:15.967465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 17 03:00:15.968738 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 17 03:00:15.968738 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 17 03:00:15.968738 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 03:00:15.968738 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 03:00:15.968738 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 17 03:00:15.968738 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:15.968738 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 373/757] 10.222.0.27 () {60 vars in 1131 bytes} [Mon Oct 17 03:00:15 2022] POST /identity/v3/users => generated 160 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:00:15.975818 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3f533-2535-45ef-851a-8239b25b59a6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:16.094883 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3f533-2535-45ef-851a-8239b25b59a6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:16.135447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abcb35-c2ae-4343-bd5d-48028308ce79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:16.136006 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abcb35-c2ae-4343-bd5d-48028308ce79 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:16.136281 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abcb35-c2ae-4343-bd5d-48028308ce79 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:16.136547 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abcb35-c2ae-4343-bd5d-48028308ce79 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:16.144693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abcb35-c2ae-4343-bd5d-48028308ce79 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:16.146113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abcb35-c2ae-4343-bd5d-48028308ce79 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:16.157320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-09abcb35-c2ae-4343-bd5d-48028308ce79 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 03:00:16.157320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:16.157320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:16.157320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:16.157320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:16.157320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:16.157320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:16.157320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:16.157320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:16.157320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:16.157320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:16.157320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:16.157320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:00:16.157320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:00:16.157320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:00:16.157320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:16.158258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:16.159286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:16.159286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:16.159286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:00:16.159286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:00:16.159286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:00:16.159286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:00:16.159286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:00:16.159286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:00:16.159286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 03:00:16.159286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:16.159286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 385/758] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 03:00:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:16.164654 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cc965-436a-437e-b86d-8ec46b33aed7 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:16.195878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:16.276424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9cc965-436a-437e-b86d-8ec46b33aed7 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:16.312434 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1b179-1339-421b-9584-e62dce924845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:16.313047 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1b179-1339-421b-9584-e62dce924845 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:16.313331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1b179-1339-421b-9584-e62dce924845 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:16.313605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1b179-1339-421b-9584-e62dce924845 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:16.314309 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1b179-1339-421b-9584-e62dce924845 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:16.314611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1b179-1339-421b-9584-e62dce924845 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:16.316287 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1b179-1339-421b-9584-e62dce924845 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:16.355940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 374/759] 10.222.0.27 () {56 vars in 1101 bytes} [Mon Oct 17 03:00:16 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:17.400208 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a6a27-8d25-4b71-b0ad-7e951d5e36c0 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:17.400746 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a6a27-8d25-4b71-b0ad-7e951d5e36c0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:17.400746 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a6a27-8d25-4b71-b0ad-7e951d5e36c0 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:17.402223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 386/760] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:17.407412 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628419aa-19b4-4ccf-8a52-db384c1f15f6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:17.407804 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628419aa-19b4-4ccf-8a52-db384c1f15f6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:17.407962 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628419aa-19b4-4ccf-8a52-db384c1f15f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:17.427115 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-628419aa-19b4-4ccf-8a52-db384c1f15f6 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:17.428613 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-628419aa-19b4-4ccf-8a52-db384c1f15f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:17.494086 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 375/761] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:17 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:17.761425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a6a27-8d25-4b71-b0ad-7e951d5e36c0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:17.797561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:17.868164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a6a27-8d25-4b71-b0ad-7e951d5e36c0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:17.905017 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a713b-1bbe-4a59-a010-a3131a094c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:17.905663 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a713b-1bbe-4a59-a010-a3131a094c27 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:17.905937 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a713b-1bbe-4a59-a010-a3131a094c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:17.906212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a713b-1bbe-4a59-a010-a3131a094c27 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:17.906796 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a713b-1bbe-4a59-a010-a3131a094c27 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:17.908184 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a713b-1bbe-4a59-a010-a3131a094c27 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:17.917397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d05a713b-1bbe-4a59-a010-a3131a094c27 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:00:17.917397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:17.917397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:00:17.917397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:00:17.917397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:00:17.917397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:00:17.917397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:00:17.917397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:17.917397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:00:17.917397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:17.917397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:17.917397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:17.917397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:17.917397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:17.917397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:17.917397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:17.917397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:17.918391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:17.918391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:17.918391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:17.918391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:17.918391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:17.918391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:17.918391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:00:17.918391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:00:17.918391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:00:17.918391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:00:17.918391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:17.918391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:17.918391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:17.918391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:17.918391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:17.918391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:17.919290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:17.919290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:17.919290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:17.919290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:17.919290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:17.919290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:17.919290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:17.919290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:17.919290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:17.919290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:17.919290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:00:17.919290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:17.919290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:00:17.919290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:17.919290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 387/762] 10.222.0.27 () {56 vars in 1126 bytes} [Mon Oct 17 03:00:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:17.924625 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628419aa-19b4-4ccf-8a52-db384c1f15f6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:17.943052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:17.982625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b1aab-f577-42cd-a886-ba386f045ccc None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:17.982936 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b1aab-f577-42cd-a886-ba386f045ccc None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:17.983215 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b1aab-f577-42cd-a886-ba386f045ccc None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:17.984653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 388/763] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:00:17.989505 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcaefc-bc7c-4b1a-9d95-b72bd44dfdcb None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:17.989785 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcaefc-bc7c-4b1a-9d95-b72bd44dfdcb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:17.990064 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcaefc-bc7c-4b1a-9d95-b72bd44dfdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:18.009349 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0bdcaefc-bc7c-4b1a-9d95-b72bd44dfdcb None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:18.010584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0bdcaefc-bc7c-4b1a-9d95-b72bd44dfdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:18.035066 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628419aa-19b4-4ccf-8a52-db384c1f15f6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:18.072217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae772fa-e26d-426e-bd5b-2da4a25949ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:18.072822 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae772fa-e26d-426e-bd5b-2da4a25949ec None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:18.073099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae772fa-e26d-426e-bd5b-2da4a25949ec None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:18.073367 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae772fa-e26d-426e-bd5b-2da4a25949ec None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:18.074114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae772fa-e26d-426e-bd5b-2da4a25949ec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:18.074416 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae772fa-e26d-426e-bd5b-2da4a25949ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:18.075823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae772fa-e26d-426e-bd5b-2da4a25949ec None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:18.081567 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 376/764] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Oct 17 03:00:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:18.087232 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae772fa-e26d-426e-bd5b-2da4a25949ec None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:18.095590 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 389/765] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:17 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:00:18.131945 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58028e5d-5781-4dcf-b983-f6b23556b56d None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:18.132264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58028e5d-5781-4dcf-b983-f6b23556b56d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:18.132509 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58028e5d-5781-4dcf-b983-f6b23556b56d None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:18.133850 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 390/766] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:00:18.138592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007b368-80a3-485b-8fb6-04e51c7c5ce3 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:18.138906 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007b368-80a3-485b-8fb6-04e51c7c5ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:18.139198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007b368-80a3-485b-8fb6-04e51c7c5ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:18.157503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8007b368-80a3-485b-8fb6-04e51c7c5ce3 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:18.158701 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8007b368-80a3-485b-8fb6-04e51c7c5ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:18.171725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:18.182697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:18.193593 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae772fa-e26d-426e-bd5b-2da4a25949ec None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:18.230300 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b08da7-9fce-4837-bf4a-45759c0ee0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:18.231212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b08da7-9fce-4837-bf4a-45759c0ee0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:18.231322 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 391/767] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:18 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:00:18.231403 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b08da7-9fce-4837-bf4a-45759c0ee0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:18.231506 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b08da7-9fce-4837-bf4a-45759c0ee0f9 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:18.232102 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b08da7-9fce-4837-bf4a-45759c0ee0f9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:18.232409 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b08da7-9fce-4837-bf4a-45759c0ee0f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:18.233782 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b08da7-9fce-4837-bf4a-45759c0ee0f9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:18.243255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 377/768] 10.222.0.27 () {56 vars in 1122 bytes} [Mon Oct 17 03:00:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:18.244705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007b368-80a3-485b-8fb6-04e51c7c5ce3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:18.347896 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007b368-80a3-485b-8fb6-04e51c7c5ce3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:18.362988 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b08da7-9fce-4837-bf4a-45759c0ee0f9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:18.383274 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab34e56-4ed3-439d-b711-745cca6b8e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:18.383887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab34e56-4ed3-439d-b711-745cca6b8e3a None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:18.384167 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab34e56-4ed3-439d-b711-745cca6b8e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:18.384440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab34e56-4ed3-439d-b711-745cca6b8e3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:18.385129 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab34e56-4ed3-439d-b711-745cca6b8e3a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:18.386492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab34e56-4ed3-439d-b711-745cca6b8e3a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:18.405833 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 392/769] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:00:18 2022] POST /identity/v3/endpoints => generated 325 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:00:18.478786 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b08da7-9fce-4837-bf4a-45759c0ee0f9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:18.500846 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab34e56-4ed3-439d-b711-745cca6b8e3a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:18.526405 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfd91e-c47e-4d2d-82c6-d74f6527c6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:18.527083 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfd91e-c47e-4d2d-82c6-d74f6527c6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:18.527366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfd91e-c47e-4d2d-82c6-d74f6527c6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:18.527631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfd91e-c47e-4d2d-82c6-d74f6527c6e9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:18.527970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:18.528221 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfd91e-c47e-4d2d-82c6-d74f6527c6e9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:18.529687 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfd91e-c47e-4d2d-82c6-d74f6527c6e9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:18.539841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-bfcfd91e-c47e-4d2d-82c6-d74f6527c6e9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:00:18.539841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:18.539841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:00:18.539841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:00:18.539841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:00:18.539841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:00:18.539841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:00:18.539841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:18.539841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:00:18.539841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:18.539841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:18.539841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:18.539841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:18.539841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:18.539841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:18.539841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:18.539841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:18.541037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:18.541037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:18.541037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:18.541037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:18.541037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:18.541037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:18.541037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:00:18.541037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:00:18.541037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:00:18.541037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:00:18.541037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:18.541037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:18.541037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:18.541037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:18.541037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:18.541037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:18.541900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:18.541900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:18.541900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:18.541900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:18.541900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:18.541900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:18.541900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:18.541900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:18.541900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:18.541900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:18.541900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:00:18.541900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:18.541900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:00:18.541900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:18.541900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 378/770] 10.222.0.27 () {56 vars in 1105 bytes} [Mon Oct 17 03:00:18 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:00:18.547305 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfd91e-c47e-4d2d-82c6-d74f6527c6e9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:18.551678 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:18.632439 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab34e56-4ed3-439d-b711-745cca6b8e3a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:18.661753 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfd91e-c47e-4d2d-82c6-d74f6527c6e9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:18.667533 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f63502-8f77-4ef4-a215-9a210de2620e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:18.668351 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f63502-8f77-4ef4-a215-9a210de2620e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:18.668796 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f63502-8f77-4ef4-a215-9a210de2620e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:18.669071 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f63502-8f77-4ef4-a215-9a210de2620e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:18.669665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f63502-8f77-4ef4-a215-9a210de2620e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:18.671071 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f63502-8f77-4ef4-a215-9a210de2620e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:18.680119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-89f63502-8f77-4ef4-a215-9a210de2620e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:00:18.680119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:18.680119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:00:18.680119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:00:18.680119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:00:18.680119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:00:18.680119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:00:18.680119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:18.680119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:00:18.680119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:18.680119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:18.680119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:18.680119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:18.680119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:18.680119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:18.680119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:18.680119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:18.683051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:18.683051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:18.683051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:18.683051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:18.683051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:18.683051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:18.683051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:00:18.683051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:00:18.683051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:00:18.683051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:00:18.683051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:18.683051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:18.683051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:18.683051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:18.683051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:18.683051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:18.684123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:18.684123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:18.684123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:18.684123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:18.684123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:18.684123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:18.684123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:18.684123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:18.684123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:18.684123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:18.684123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:00:18.684123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:18.684123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:00:18.684123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:18.684123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 393/771] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 03:00:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:00:18.686849 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f63502-8f77-4ef4-a215-9a210de2620e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:18.704404 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678cf8d-2c62-4e9c-823c-706b5aaf00d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:18.705047 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678cf8d-2c62-4e9c-823c-706b5aaf00d1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:18.705329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678cf8d-2c62-4e9c-823c-706b5aaf00d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:18.705602 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678cf8d-2c62-4e9c-823c-706b5aaf00d1 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:18.706235 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678cf8d-2c62-4e9c-823c-706b5aaf00d1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:18.706539 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678cf8d-2c62-4e9c-823c-706b5aaf00d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:18.708653 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678cf8d-2c62-4e9c-823c-706b5aaf00d1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:18.713462 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 379/772] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:18.719085 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678cf8d-2c62-4e9c-823c-706b5aaf00d1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:18.772469 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:18.793733 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:18.796075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f63502-8f77-4ef4-a215-9a210de2620e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:18.831065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678cf8d-2c62-4e9c-823c-706b5aaf00d1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:18.831672 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988503d-fa7f-4039-831a-3b29bc339a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:18.833199 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988503d-fa7f-4039-831a-3b29bc339a81 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:18.833555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988503d-fa7f-4039-831a-3b29bc339a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:18.833749 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988503d-fa7f-4039-831a-3b29bc339a81 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:18.834956 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988503d-fa7f-4039-831a-3b29bc339a81 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:18.834956 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988503d-fa7f-4039-831a-3b29bc339a81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:18.836070 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988503d-fa7f-4039-831a-3b29bc339a81 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:18.846251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 394/773] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Oct 17 03:00:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:18.847659 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988503d-fa7f-4039-831a-3b29bc339a81 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:18.866850 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562e139-fb01-4342-ae9f-3f62f60ca8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:18.867502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562e139-fb01-4342-ae9f-3f62f60ca8dc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:18.867785 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562e139-fb01-4342-ae9f-3f62f60ca8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:18.868061 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562e139-fb01-4342-ae9f-3f62f60ca8dc None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:18.868695 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562e139-fb01-4342-ae9f-3f62f60ca8dc None admin] RBAC: Adding query filter params (type=image) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:18.869003 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562e139-fb01-4342-ae9f-3f62f60ca8dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:18.870391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562e139-fb01-4342-ae9f-3f62f60ca8dc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:18.876420 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 380/774] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:18.883030 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562e139-fb01-4342-ae9f-3f62f60ca8dc None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:18.962034 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988503d-fa7f-4039-831a-3b29bc339a81 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:18.989889 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562e139-fb01-4342-ae9f-3f62f60ca8dc None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:18.998611 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0add1-d210-4661-8270-ea68bdf2d8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:18.999255 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0add1-d210-4661-8270-ea68bdf2d8db None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:18.999541 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0add1-d210-4661-8270-ea68bdf2d8db None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:18.999814 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0add1-d210-4661-8270-ea68bdf2d8db None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:19.000445 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0add1-d210-4661-8270-ea68bdf2d8db None admin] RBAC: Adding query filter params (type=compute) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:19.000750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0add1-d210-4661-8270-ea68bdf2d8db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:19.002112 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0add1-d210-4661-8270-ea68bdf2d8db None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:19.008139 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 395/775] 10.222.0.27 () {56 vars in 1112 bytes} [Mon Oct 17 03:00:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:19.013441 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0add1-d210-4661-8270-ea68bdf2d8db None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:19.023505 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:19.025628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:19.031263 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1b10b-e3de-459c-80ea-769a20465253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:19.031985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1b10b-e3de-459c-80ea-769a20465253 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:19.032271 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1b10b-e3de-459c-80ea-769a20465253 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:19.032547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1b10b-e3de-459c-80ea-769a20465253 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:19.033361 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1b10b-e3de-459c-80ea-769a20465253 None admin] RBAC: Adding query filter params (interface=public, service_id=fb31b610e8b54c5b92bd6bd69aca3dd1, region_id=RegionOne) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:19.033673 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1b10b-e3de-459c-80ea-769a20465253 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:19.035912 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1b10b-e3de-459c-80ea-769a20465253 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:19.042582 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 381/776] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Oct 17 03:00:18 2022] GET /identity/v3/endpoints?service_id=fb31b610e8b54c5b92bd6bd69aca3dd1&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:19.048172 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1b10b-e3de-459c-80ea-769a20465253 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:19.115245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0add1-d210-4661-8270-ea68bdf2d8db None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:19.151785 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543938bd-f3b4-4935-b6b3-7085c7393126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:19.152125 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1b10b-e3de-459c-80ea-769a20465253 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:19.152306 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543938bd-f3b4-4935-b6b3-7085c7393126 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:19.152584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543938bd-f3b4-4935-b6b3-7085c7393126 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:19.152858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543938bd-f3b4-4935-b6b3-7085c7393126 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:19.153522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543938bd-f3b4-4935-b6b3-7085c7393126 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:19.154889 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543938bd-f3b4-4935-b6b3-7085c7393126 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:19.174962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 396/777] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:00:19 2022] POST /identity/v3/endpoints => generated 334 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:00:19.205836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ead80-4c9a-4dc2-81c8-777578d5a7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:19.206462 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ead80-4c9a-4dc2-81c8-777578d5a7dc None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:19.206743 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ead80-4c9a-4dc2-81c8-777578d5a7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:19.207308 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ead80-4c9a-4dc2-81c8-777578d5a7dc None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:19.207719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ead80-4c9a-4dc2-81c8-777578d5a7dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:19.209122 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ead80-4c9a-4dc2-81c8-777578d5a7dc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:19.217020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 382/778] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Oct 17 03:00:19 2022] GET /identity/v3/services => generated 1957 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:00:20.625815 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393df9ab-accb-4d1e-8221-ab7a0a77c02e None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:20.626333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393df9ab-accb-4d1e-8221-ab7a0a77c02e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:20.626444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393df9ab-accb-4d1e-8221-ab7a0a77c02e None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:20.627923 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 397/779] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:20.634206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908690d1-1e09-4bad-b9f0-f9ea1e67e8b1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:20.634583 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908690d1-1e09-4bad-b9f0-f9ea1e67e8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:20.634758 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908690d1-1e09-4bad-b9f0-f9ea1e67e8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:20.653740 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-908690d1-1e09-4bad-b9f0-f9ea1e67e8b1 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:20.654966 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-908690d1-1e09-4bad-b9f0-f9ea1e67e8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:20.706151 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:20.725543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 383/780] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:21.001384 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393df9ab-accb-4d1e-8221-ab7a0a77c02e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:21.070660 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:21.107822 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393df9ab-accb-4d1e-8221-ab7a0a77c02e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:21.141248 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff45a53-b232-4617-93a8-edc7155a74d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:21.141874 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff45a53-b232-4617-93a8-edc7155a74d3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:21.142147 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff45a53-b232-4617-93a8-edc7155a74d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:21.142415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff45a53-b232-4617-93a8-edc7155a74d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:21.143525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff45a53-b232-4617-93a8-edc7155a74d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:21.145611 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff45a53-b232-4617-93a8-edc7155a74d3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:21.148299 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 398/781] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:21.153065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908690d1-1e09-4bad-b9f0-f9ea1e67e8b1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:21.255691 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908690d1-1e09-4bad-b9f0-f9ea1e67e8b1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:21.296057 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd759dd-4cc2-4536-8592-5381af6dc933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:21.296699 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd759dd-4cc2-4536-8592-5381af6dc933 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:21.296975 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd759dd-4cc2-4536-8592-5381af6dc933 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:21.297251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd759dd-4cc2-4536-8592-5381af6dc933 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:21.297969 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd759dd-4cc2-4536-8592-5381af6dc933 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:21.299520 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd759dd-4cc2-4536-8592-5381af6dc933 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:21.338953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 384/782] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Oct 17 03:00:21 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:00:21.404885 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d296f7d-e7a6-4fe2-bfb0-505a00900ca4 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:21.405233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d296f7d-e7a6-4fe2-bfb0-505a00900ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:21.405440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d296f7d-e7a6-4fe2-bfb0-505a00900ca4 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:21.407092 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 399/783] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:21.413245 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fc7b5-f8cb-474e-9b80-8dba9ed54e8c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:21.413552 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fc7b5-f8cb-474e-9b80-8dba9ed54e8c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:21.413809 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fc7b5-f8cb-474e-9b80-8dba9ed54e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:21.433771 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-588fc7b5-f8cb-474e-9b80-8dba9ed54e8c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:21.435056 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-588fc7b5-f8cb-474e-9b80-8dba9ed54e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:21.451418 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6c8d1-0ba6-4043-accc-cae9b2f014e3 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:21.451738 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6c8d1-0ba6-4043-accc-cae9b2f014e3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:21.451974 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6c8d1-0ba6-4043-accc-cae9b2f014e3 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:21.453479 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 400/784] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:21.459884 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9202ac-a44b-4863-af9e-8a9578308c7a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:21.460221 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9202ac-a44b-4863-af9e-8a9578308c7a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:21.460502 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9202ac-a44b-4863-af9e-8a9578308c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:21.471487 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:21.480253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6d9202ac-a44b-4863-af9e-8a9578308c7a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:21.481513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6d9202ac-a44b-4863-af9e-8a9578308c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:21.520966 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 385/785] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:21.559982 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 401/786] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:00:21.789072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588fc7b5-f8cb-474e-9b80-8dba9ed54e8c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:21.828973 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9202ac-a44b-4863-af9e-8a9578308c7a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:21.898975 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588fc7b5-f8cb-474e-9b80-8dba9ed54e8c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:21.900378 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:21.938636 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c31ddb-e203-4c69-8a79-15e1715e2657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:21.939306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c31ddb-e203-4c69-8a79-15e1715e2657 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:21.939586 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c31ddb-e203-4c69-8a79-15e1715e2657 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:21.939859 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c31ddb-e203-4c69-8a79-15e1715e2657 None admin] PATH_INFO: `/v3/services/image` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:21.940444 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c31ddb-e203-4c69-8a79-15e1715e2657 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:21.941031 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9202ac-a44b-4863-af9e-8a9578308c7a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:21.941879 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c31ddb-e203-4c69-8a79-15e1715e2657 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:21.950661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-24c31ddb-e203-4c69-8a79-15e1715e2657 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:00:21.950661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:21.950661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:00:21.950661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:00:21.950661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:00:21.950661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:00:21.950661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:00:21.950661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:21.950661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:00:21.950661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:21.950661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:21.950661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:21.950661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:21.950661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:21.950661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:21.950661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:21.950661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:21.951882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:21.951882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:21.951882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:21.951882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:21.951882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:21.951882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:21.951882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:00:21.951882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:00:21.951882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:00:21.951882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:00:21.951882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:21.951882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:21.951882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:21.951882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:21.951882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:21.951882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:21.955827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:21.955827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:21.955827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:21.955827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:21.955827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:21.955827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:21.955827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:21.955827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:21.955827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:21.955827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:21.955827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:00:21.955827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:00:21.955827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:00:21.955827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:21.955827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 386/787] 10.222.0.27 () {56 vars in 1105 bytes} [Mon Oct 17 03:00:21 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:21.958005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c31ddb-e203-4c69-8a79-15e1715e2657 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:21.983372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c915826-1013-4353-99d0-f10580b78d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:21.984002 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c915826-1013-4353-99d0-f10580b78d86 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:21.984278 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c915826-1013-4353-99d0-f10580b78d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:21.984738 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c915826-1013-4353-99d0-f10580b78d86 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:21.991510 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:21.995491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c915826-1013-4353-99d0-f10580b78d86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:00:22.000212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-3c915826-1013-4353-99d0-f10580b78d86 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 03:00:22.000212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:22.000212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:22.000212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:22.000212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:22.000212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:22.000212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:22.000212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:22.000212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:22.000212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:22.000212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:22.000212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:22.000212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:00:22.000212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:00:22.000212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:00:22.000212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:00:22.001145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:00:22.001145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:00:22.001145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:00:22.001145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:00:22.001145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:22.001145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:22.001145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:22.001145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:22.001145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:22.001145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:22.001145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:22.001145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:22.001145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:22.001145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:22.001145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:22.001145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:22.002026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:22.002026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:22.002026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:22.002026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:22.002026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:00:22.002026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:00:22.002026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:00:22.002026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:00:22.002026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:00:22.002026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:00:22.002026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 03:00:22.002026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:22.002026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 402/788] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Oct 17 03:00:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:22.007456 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c915826-1013-4353-99d0-f10580b78d86 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:22.069115 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c31ddb-e203-4c69-8a79-15e1715e2657 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:22.104734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bc851-c8d1-430f-9dc1-20069fbf803d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:22.105346 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bc851-c8d1-430f-9dc1-20069fbf803d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:22.105623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bc851-c8d1-430f-9dc1-20069fbf803d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:22.105894 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bc851-c8d1-430f-9dc1-20069fbf803d None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:22.106521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bc851-c8d1-430f-9dc1-20069fbf803d None admin] RBAC: Adding query filter params (name=image) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:22.106833 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bc851-c8d1-430f-9dc1-20069fbf803d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:22.107964 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c915826-1013-4353-99d0-f10580b78d86 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:22.108222 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bc851-c8d1-430f-9dc1-20069fbf803d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:22.118330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 387/789] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:22.119410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bc851-c8d1-430f-9dc1-20069fbf803d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:22.135340 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:22.147674 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35204f-7e78-4bae-a225-56e6193de698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:22.148293 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35204f-7e78-4bae-a225-56e6193de698 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:22.148575 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35204f-7e78-4bae-a225-56e6193de698 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:22.148852 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35204f-7e78-4bae-a225-56e6193de698 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:22.149519 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35204f-7e78-4bae-a225-56e6193de698 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:22.149828 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35204f-7e78-4bae-a225-56e6193de698 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:22.151223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da35204f-7e78-4bae-a225-56e6193de698 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:22.164321 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 403/790] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Oct 17 03:00:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:22.167956 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35204f-7e78-4bae-a225-56e6193de698 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:22.229601 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:22.237161 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bc851-c8d1-430f-9dc1-20069fbf803d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:22.271701 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35204f-7e78-4bae-a225-56e6193de698 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:22.273232 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5a9df-32a6-4bf1-9825-a91255f58a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:22.273842 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5a9df-32a6-4bf1-9825-a91255f58a77 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:22.277539 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5a9df-32a6-4bf1-9825-a91255f58a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:22.277539 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5a9df-32a6-4bf1-9825-a91255f58a77 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:22.277539 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5a9df-32a6-4bf1-9825-a91255f58a77 None admin] RBAC: Adding query filter params (type=image) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:22.277539 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5a9df-32a6-4bf1-9825-a91255f58a77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:22.277539 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5a9df-32a6-4bf1-9825-a91255f58a77 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:22.283014 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 388/791] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:22.290107 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5a9df-32a6-4bf1-9825-a91255f58a77 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:22.315800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cb89c-3dd8-49f6-9076-8e0436c0764a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:22.316588 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cb89c-3dd8-49f6-9076-8e0436c0764a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:22.316939 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cb89c-3dd8-49f6-9076-8e0436c0764a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:22.317284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cb89c-3dd8-49f6-9076-8e0436c0764a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:22.328256 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21cb89c-3dd8-49f6-9076-8e0436c0764a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:22.329625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21cb89c-3dd8-49f6-9076-8e0436c0764a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:22.344498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e21cb89c-3dd8-49f6-9076-8e0436c0764a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:22.344498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:22.344498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:22.344498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:22.344498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:22.344498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:22.344498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:22.344498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:22.344498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:22.344498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:22.344498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:22.344498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:22.344498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:00:22.344498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:00:22.344498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:00:22.344498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:00:22.345444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:22.345444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:22.345444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:22.345444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:22.345444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:22.345444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:22.345444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:22.345444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:22.345444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:22.345444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:22.345444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:22.345444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:22.345444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:22.345444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:22.345444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:22.345444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:22.345444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:00:22.346333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:00:22.346333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:22.346333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:22.346333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 404/792] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:00:22.351442 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cb89c-3dd8-49f6-9076-8e0436c0764a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:22.393478 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:22.400819 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5a9df-32a6-4bf1-9825-a91255f58a77 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:22.434780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b2686-d20a-4a5b-9504-1a35683833dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:22.435522 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b2686-d20a-4a5b-9504-1a35683833dc None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:22.435798 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b2686-d20a-4a5b-9504-1a35683833dc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:22.436070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b2686-d20a-4a5b-9504-1a35683833dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:22.436729 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b2686-d20a-4a5b-9504-1a35683833dc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:22.438219 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b2686-d20a-4a5b-9504-1a35683833dc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:22.453778 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cb89c-3dd8-49f6-9076-8e0436c0764a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:22.464258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 389/793] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:00:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:00:22.501871 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a48a2-78ba-4cf6-9909-043857cde957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:22.502495 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a48a2-78ba-4cf6-9909-043857cde957 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:22.502771 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a48a2-78ba-4cf6-9909-043857cde957 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:22.503095 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a48a2-78ba-4cf6-9909-043857cde957 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:22.503747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a48a2-78ba-4cf6-9909-043857cde957 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:22.504641 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a48a2-78ba-4cf6-9909-043857cde957 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:22.506205 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a48a2-78ba-4cf6-9909-043857cde957 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:22.520938 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 405/794] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:22.526550 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b2686-d20a-4a5b-9504-1a35683833dc None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:22.546307 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:22.643685 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b2686-d20a-4a5b-9504-1a35683833dc None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:22.680418 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad5719-cf89-4466-a883-0e716a0e8c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:22.681052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad5719-cf89-4466-a883-0e716a0e8c3c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:22.681328 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad5719-cf89-4466-a883-0e716a0e8c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:22.681598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad5719-cf89-4466-a883-0e716a0e8c3c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:22.689061 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad5719-cf89-4466-a883-0e716a0e8c3c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:22.690508 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad5719-cf89-4466-a883-0e716a0e8c3c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:22.702711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-51ad5719-cf89-4466-a883-0e716a0e8c3c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 03:00:22.702711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:22.702711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:22.702711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:22.702711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:22.702711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:22.702711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:22.702711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:22.702711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:22.702711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:22.702711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:22.702711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:22.702711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:00:22.702711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:00:22.702711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:00:22.702711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:22.703712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:22.705264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:22.705264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:22.705264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:00:22.705264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:00:22.705264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:00:22.705264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:00:22.705264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:00:22.705264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:00:22.705264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 03:00:22.705264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:22.705264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 390/795] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Oct 17 03:00:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:22.710425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a48a2-78ba-4cf6-9909-043857cde957 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:22.804194 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:22.812257 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a48a2-78ba-4cf6-9909-043857cde957 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:22.846274 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a20fe-309c-4b7a-8c03-2df0f8e2e13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:22.847058 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a20fe-309c-4b7a-8c03-2df0f8e2e13c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:22.847230 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a20fe-309c-4b7a-8c03-2df0f8e2e13c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:22.847495 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9a20fe-309c-4b7a-8c03-2df0f8e2e13c None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:22.848439 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a20fe-309c-4b7a-8c03-2df0f8e2e13c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:22.848743 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a20fe-309c-4b7a-8c03-2df0f8e2e13c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:22.850156 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9a20fe-309c-4b7a-8c03-2df0f8e2e13c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:22.872608 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 406/796] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Oct 17 03:00:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:22.873816 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad5719-cf89-4466-a883-0e716a0e8c3c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:22.955384 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:22.978770 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad5719-cf89-4466-a883-0e716a0e8c3c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:23.014242 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a9b72-17b3-4663-b374-213f18533e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:23.014916 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a9b72-17b3-4663-b374-213f18533e4e None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:23.015184 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a9b72-17b3-4663-b374-213f18533e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:23.015454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a9b72-17b3-4663-b374-213f18533e4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:23.026572 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a9b72-17b3-4663-b374-213f18533e4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:23.028105 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a9b72-17b3-4663-b374-213f18533e4e None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:23.042182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-fd7a9b72-17b3-4663-b374-213f18533e4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:23.042182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:23.042182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:23.042182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:23.042182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:23.042182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:23.042182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:23.042182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:23.042182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:23.042182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:23.042182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:23.042182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:23.042182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:00:23.042182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:00:23.042182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:00:23.042182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:00:23.043382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:23.043382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:23.043382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:23.043382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:23.043382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:23.043382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:23.043382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:23.043382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:23.043382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:23.043382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:23.043382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:23.043382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:23.043382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:23.043382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:23.043382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:23.043382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:23.043382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:00:23.044298 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:00:23.044298 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:23.044298 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:23.044298 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 391/797] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:23.050508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a20fe-309c-4b7a-8c03-2df0f8e2e13c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:23.153857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9a20fe-309c-4b7a-8c03-2df0f8e2e13c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:23.188938 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a533e5-d8ac-473c-b440-c44f6bf9b5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:23.189569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a533e5-d8ac-473c-b440-c44f6bf9b5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:23.189851 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a533e5-d8ac-473c-b440-c44f6bf9b5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:23.190123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a533e5-d8ac-473c-b440-c44f6bf9b5c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:23.190765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a533e5-d8ac-473c-b440-c44f6bf9b5c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:23.191116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a533e5-d8ac-473c-b440-c44f6bf9b5c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:23.192751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a533e5-d8ac-473c-b440-c44f6bf9b5c5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:23.208934 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 407/798] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:23.210574 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a9b72-17b3-4663-b374-213f18533e4e None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:23.325832 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a9b72-17b3-4663-b374-213f18533e4e None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:23.352156 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:23.361471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf651c-4cf3-4cb2-9dee-ee6576a2062c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:23.362130 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf651c-4cf3-4cb2-9dee-ee6576a2062c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:23.362414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf651c-4cf3-4cb2-9dee-ee6576a2062c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:23.362688 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf651c-4cf3-4cb2-9dee-ee6576a2062c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:23.369448 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf651c-4cf3-4cb2-9dee-ee6576a2062c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:23.370931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf651c-4cf3-4cb2-9dee-ee6576a2062c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:23.383364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-0faf651c-4cf3-4cb2-9dee-ee6576a2062c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:00:23.383364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:23.383364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:23.383364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:23.383364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:23.383364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:23.383364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:23.383364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:23.383364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:23.383364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:23.383364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:23.383364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:23.383364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:00:23.383364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:00:23.383364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:00:23.384410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:00:23.384410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:23.384410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:23.384410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:23.384410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:23.384410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:23.384410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:23.384410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:23.384410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:23.384410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:23.384410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:23.384410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:23.384410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:23.384410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:23.384410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:23.384410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:23.384410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:23.390365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:00:23.390365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:00:23.390365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:00:23.390365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:00:23.390365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:00:23.390365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:00:23.390365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:00:23.390365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:23.390365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 392/799] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 03:00:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:00:23.392732 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a533e5-d8ac-473c-b440-c44f6bf9b5c5 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:23.409693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:23.501934 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a533e5-d8ac-473c-b440-c44f6bf9b5c5 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:23.536838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4bd39-5ade-4c1a-9757-1202149ab2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:23.537501 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4bd39-5ade-4c1a-9757-1202149ab2a5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:23.537785 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4bd39-5ade-4c1a-9757-1202149ab2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:23.538057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4bd39-5ade-4c1a-9757-1202149ab2a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:23.538741 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4bd39-5ade-4c1a-9757-1202149ab2a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:23.539075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4bd39-5ade-4c1a-9757-1202149ab2a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:23.540492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4bd39-5ade-4c1a-9757-1202149ab2a5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:23.552979 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 408/800] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 03:00:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:23.558885 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf651c-4cf3-4cb2-9dee-ee6576a2062c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:23.579887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c23cb5-a658-408c-8485-5cd2a025416e None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:23.580167 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c23cb5-a658-408c-8485-5cd2a025416e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:23.580443 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c23cb5-a658-408c-8485-5cd2a025416e None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:23.581874 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 409/801] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:23.587062 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298045a8-ffc2-40c3-9d7e-62d3b2a48f21 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:23.587347 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298045a8-ffc2-40c3-9d7e-62d3b2a48f21 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:23.587626 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298045a8-ffc2-40c3-9d7e-62d3b2a48f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:23.605578 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-298045a8-ffc2-40c3-9d7e-62d3b2a48f21 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:23.606743 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-298045a8-ffc2-40c3-9d7e-62d3b2a48f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:23.667590 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf651c-4cf3-4cb2-9dee-ee6576a2062c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:23.682317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 410/802] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:00:23.703495 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5607d36-9716-4326-ba0f-ef78bd390328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:23.704155 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5607d36-9716-4326-ba0f-ef78bd390328 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:23.704435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5607d36-9716-4326-ba0f-ef78bd390328 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:23.704708 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5607d36-9716-4326-ba0f-ef78bd390328 None admin] PATH_INFO: `/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/01637fc07ba846aebcb42325bed46216/roles/355fbb07456348ac8f6ae42b3dd96690` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:23.706909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5607d36-9716-4326-ba0f-ef78bd390328 None admin] RBAC: Authorizing `identity:create_grant(project_id=89b0ee95955f465eb9ee5b8ae337d9c2, user_id=01637fc07ba846aebcb42325bed46216, role_id=355fbb07456348ac8f6ae42b3dd96690)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:23.709334 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5607d36-9716-4326-ba0f-ef78bd390328 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:23.724677 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 393/803] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 03:00:23 2022] PUT /identity/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/01637fc07ba846aebcb42325bed46216/roles/355fbb07456348ac8f6ae42b3dd96690 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:00:23.952318 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298045a8-ffc2-40c3-9d7e-62d3b2a48f21 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:23.954483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:24.051818 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298045a8-ffc2-40c3-9d7e-62d3b2a48f21 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:24.085962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d4174-9f27-4096-9ced-1b560f17f1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:24.086984 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d4174-9f27-4096-9ced-1b560f17f1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:24.087475 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d4174-9f27-4096-9ced-1b560f17f1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:24.087985 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d4174-9f27-4096-9ced-1b560f17f1d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:24.089273 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d4174-9f27-4096-9ced-1b560f17f1d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:24.090897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d4174-9f27-4096-9ced-1b560f17f1d5 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:24.092955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 411/804] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:24.099113 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5607d36-9716-4326-ba0f-ef78bd390328 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:24.150493 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:24.205638 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5607d36-9716-4326-ba0f-ef78bd390328 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:24.242602 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2d9ee-f4d7-49ad-9398-2c760f73873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:24.243397 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2d9ee-f4d7-49ad-9398-2c760f73873b None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:24.243774 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2d9ee-f4d7-49ad-9398-2c760f73873b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:24.244146 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2d9ee-f4d7-49ad-9398-2c760f73873b None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:24.244886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2d9ee-f4d7-49ad-9398-2c760f73873b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:24.246408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2d9ee-f4d7-49ad-9398-2c760f73873b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:24.278900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 394/805] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 03:00:24 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:00:24.692113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8b18c-d877-4de8-bccc-ccc81a046f19 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:24.692529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8b18c-d877-4de8-bccc-ccc81a046f19 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:24.692717 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8b18c-d877-4de8-bccc-ccc81a046f19 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:24.694026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 412/806] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:24.699138 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198d239-1f5e-4918-a396-b5e71a1a6644 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:24.699418 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198d239-1f5e-4918-a396-b5e71a1a6644 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:24.699679 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198d239-1f5e-4918-a396-b5e71a1a6644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:24.717574 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a198d239-1f5e-4918-a396-b5e71a1a6644 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:24.718751 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a198d239-1f5e-4918-a396-b5e71a1a6644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:24.787949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 395/807] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:25.052801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8b18c-d877-4de8-bccc-ccc81a046f19 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:25.126335 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:25.159469 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8b18c-d877-4de8-bccc-ccc81a046f19 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:25.196500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6cebe-612c-457c-a29c-06555f99d059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:25.197121 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6cebe-612c-457c-a29c-06555f99d059 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:25.197401 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6cebe-612c-457c-a29c-06555f99d059 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:25.197676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6cebe-612c-457c-a29c-06555f99d059 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:25.198365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6cebe-612c-457c-a29c-06555f99d059 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:25.199759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6cebe-612c-457c-a29c-06555f99d059 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:25.207507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.sql.core [None req-e2a6cebe-612c-457c-a29c-06555f99d059 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 03:00:25.207507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 03:00:25.207507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [parameters: {'id': '09ad2e783e8845ceba235e164a7dff6d', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 03:00:25.207507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81582) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 03:00:25.215072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e2a6cebe-612c-457c-a29c-06555f99d059 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 03:00:25.215072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:25.215072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 03:00:25.215072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 03:00:25.215072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 03:00:25.215072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 03:00:25.215072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 03:00:25.215072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 03:00:25.215072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 03:00:25.215072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application conn.query(q) Oct 17 03:00:25.215072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 03:00:25.215072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 03:00:25.215072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 03:00:25.215072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result.read() Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 03:00:25.215955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 03:00:25.216994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application next(self.gen) Oct 17 03:00:25.216994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 03:00:25.216994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application yield resource Oct 17 03:00:25.216994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 03:00:25.216994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application next(self.gen) Oct 17 03:00:25.216994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 03:00:25.216994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 03:00:25.216994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 03:00:25.216994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 03:00:25.216994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 03:00:25.216994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise self.value Oct 17 03:00:25.216994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 03:00:25.216994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 03:00:25.216994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 03:00:25.216994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application session.commit() Oct 17 03:00:25.216994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 03:00:25.216994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 03:00:25.217932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 03:00:25.217932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 03:00:25.217932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 03:00:25.217932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.session.flush() Oct 17 03:00:25.217932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 03:00:25.217932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 03:00:25.217932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 03:00:25.217932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 03:00:25.217932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 03:00:25.217932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application compat.raise_( Oct 17 03:00:25.217932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 03:00:25.217932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception Oct 17 03:00:25.217932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 03:00:25.217932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 03:00:25.217932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 03:00:25.217932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 03:00:25.217932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 03:00:25.218856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 03:00:25.218856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 03:00:25.218856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 03:00:25.218856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 03:00:25.218856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 03:00:25.218856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 03:00:25.218856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 03:00:25.218856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 03:00:25.218856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 03:00:25.218856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 03:00:25.218856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 03:00:25.218856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 03:00:25.218856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 03:00:25.218856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 03:00:25.218856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 03:00:25.222712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 03:00:25.222712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception Oct 17 03:00:25.222712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 03:00:25.222712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 03:00:25.222712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 03:00:25.222712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 03:00:25.222712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 03:00:25.222712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 03:00:25.222712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 03:00:25.222712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application conn.query(q) Oct 17 03:00:25.222712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 03:00:25.222712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 03:00:25.222712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 03:00:25.222712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application result.read() Oct 17 03:00:25.222712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 03:00:25.222712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 03:00:25.222712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 03:00:25.224016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 03:00:25.224016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 03:00:25.224016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 03:00:25.224016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 03:00:25.224016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 03:00:25.224016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 03:00:25.224016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 03:00:25.224016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [parameters: {'id': '09ad2e783e8845ceba235e164a7dff6d', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 03:00:25.224016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 03:00:25.224016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:25.224016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:25.224016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:25.224016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:25.224016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 03:00:25.224016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 03:00:25.225094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 03:00:25.225094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 03:00:25.225094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 17 03:00:25.225094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:25.225094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:00:25.225094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:25.225094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:25.225094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:25.225094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:25.225094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:25.225094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:25.225094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:25.225094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:25.225094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:25.225094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:25.225094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:25.225094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:25.226434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 17 03:00:25.226434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 17 03:00:25.226434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:25.226434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:25.226434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 17 03:00:25.226434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 17 03:00:25.226434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:25.226434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:25.226434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 03:00:25.226434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 03:00:25.226434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 03:00:25.226434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:25.226434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 413/808] 10.222.0.27 () {60 vars in 1137 bytes} [Mon Oct 17 03:00:25 2022] POST /identity/v3/domains => generated 190 bytes in 168 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:25.231121 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198d239-1f5e-4918-a396-b5e71a1a6644 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:25.267153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:25.338994 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198d239-1f5e-4918-a396-b5e71a1a6644 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:25.372591 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb671a6-b726-4ec7-b2ab-27ddd128e871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:25.373222 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb671a6-b726-4ec7-b2ab-27ddd128e871 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:25.373505 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb671a6-b726-4ec7-b2ab-27ddd128e871 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:25.373781 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb671a6-b726-4ec7-b2ab-27ddd128e871 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:25.384851 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb671a6-b726-4ec7-b2ab-27ddd128e871 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:25.386310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb671a6-b726-4ec7-b2ab-27ddd128e871 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:25.401357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-abb671a6-b726-4ec7-b2ab-27ddd128e871 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:25.401357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:25.401357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:25.401357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:25.401357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:25.401357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:25.401357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:25.401357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:25.401357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:25.401357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:25.401357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:25.401357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:25.401357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:00:25.401357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:00:25.401357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:00:25.401357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:00:25.402339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:25.402339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:25.402339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:25.402339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:25.402339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:25.402339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:25.402339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:25.402339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:25.402339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:25.402339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:25.402339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:25.402339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:25.402339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:25.402339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:25.402339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:25.402339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:25.402339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:00:25.408157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:00:25.408157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:25.408157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:25.408157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 396/809] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:25.409897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6cebe-612c-457c-a29c-06555f99d059 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:25.511063 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6cebe-612c-457c-a29c-06555f99d059 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:25.545120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de55548-6dcb-404d-b324-c9a0f612b64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:25.545734 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de55548-6dcb-404d-b324-c9a0f612b64a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:25.546011 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de55548-6dcb-404d-b324-c9a0f612b64a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:25.546281 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de55548-6dcb-404d-b324-c9a0f612b64a None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:25.546947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de55548-6dcb-404d-b324-c9a0f612b64a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:25.547249 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de55548-6dcb-404d-b324-c9a0f612b64a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:25.548602 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de55548-6dcb-404d-b324-c9a0f612b64a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:25.550053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:25.565033 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 414/810] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:25.926975 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721d0bd-a28b-49aa-936b-836b671eecf1 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:25.927257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721d0bd-a28b-49aa-936b-836b671eecf1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:25.927530 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721d0bd-a28b-49aa-936b-836b671eecf1 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:25.928865 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 397/811] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:25.934228 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c8193-e8d5-4e96-a761-3c73084f9ffd None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:25.934511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c8193-e8d5-4e96-a761-3c73084f9ffd None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:25.934782 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c8193-e8d5-4e96-a761-3c73084f9ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:25.953160 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d98c8193-e8d5-4e96-a761-3c73084f9ffd None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:25.954320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d98c8193-e8d5-4e96-a761-3c73084f9ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:26.018488 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 415/812] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:26.284932 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721d0bd-a28b-49aa-936b-836b671eecf1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:26.374710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:26.391771 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721d0bd-a28b-49aa-936b-836b671eecf1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:26.424455 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07752cf8-7dda-47c1-a61e-b214c8ac646d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:26.425077 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07752cf8-7dda-47c1-a61e-b214c8ac646d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:26.425353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07752cf8-7dda-47c1-a61e-b214c8ac646d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:26.425624 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07752cf8-7dda-47c1-a61e-b214c8ac646d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:26.436581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07752cf8-7dda-47c1-a61e-b214c8ac646d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:00:26.441248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-07752cf8-7dda-47c1-a61e-b214c8ac646d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 03:00:26.441248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:26.441248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:26.441248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:26.441248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:26.441248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:26.441248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:26.441248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:26.441248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:26.441248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:26.441248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:26.441248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:26.441248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:00:26.441248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:00:26.441248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:00:26.441248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:00:26.442598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:00:26.442598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:00:26.442598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:00:26.442598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:00:26.442598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:26.442598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:26.442598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:26.442598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:26.442598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:26.442598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:26.442598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:26.442598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:26.442598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:26.442598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:26.442598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:26.442598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:26.443596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:26.443596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:26.443596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:26.443596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:26.443596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:00:26.443596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:00:26.443596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:00:26.443596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:00:26.443596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:00:26.443596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:00:26.443596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 03:00:26.443596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:26.447015 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 398/813] 10.222.0.27 () {56 vars in 1120 bytes} [Mon Oct 17 03:00:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:26.448332 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c8193-e8d5-4e96-a761-3c73084f9ffd None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:26.450098 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c35ca-994d-4a18-a37b-2a66242ef570 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:26.450384 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c35ca-994d-4a18-a37b-2a66242ef570 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:26.450661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c35ca-994d-4a18-a37b-2a66242ef570 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:26.451968 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 399/814] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:26.456707 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5328d0-4e89-4ff0-8f88-4c989347dbc6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:26.457020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5328d0-4e89-4ff0-8f88-4c989347dbc6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:26.457297 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5328d0-4e89-4ff0-8f88-4c989347dbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:26.475350 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-df5328d0-4e89-4ff0-8f88-4c989347dbc6 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:26.476536 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-df5328d0-4e89-4ff0-8f88-4c989347dbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:26.542570 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 400/815] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:00:26.546328 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c8193-e8d5-4e96-a761-3c73084f9ffd None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:26.569331 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:26.580287 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf04652-7b71-48aa-a91b-cb06cb26313e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:26.580897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf04652-7b71-48aa-a91b-cb06cb26313e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:26.581176 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf04652-7b71-48aa-a91b-cb06cb26313e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:26.581448 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf04652-7b71-48aa-a91b-cb06cb26313e None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:26.582617 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf04652-7b71-48aa-a91b-cb06cb26313e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:26.583009 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf04652-7b71-48aa-a91b-cb06cb26313e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:26.584395 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf04652-7b71-48aa-a91b-cb06cb26313e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:26.592095 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 416/816] 10.222.0.27 () {56 vars in 1115 bytes} [Mon Oct 17 03:00:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:26.598331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5328d0-4e89-4ff0-8f88-4c989347dbc6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:26.672569 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:26.695364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5328d0-4e89-4ff0-8f88-4c989347dbc6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:26.728565 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc72f69-a3e8-4169-8d6e-67271aeca381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:26.729186 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc72f69-a3e8-4169-8d6e-67271aeca381 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:26.729470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc72f69-a3e8-4169-8d6e-67271aeca381 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:26.729746 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc72f69-a3e8-4169-8d6e-67271aeca381 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:26.739731 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc72f69-a3e8-4169-8d6e-67271aeca381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:26.741173 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc72f69-a3e8-4169-8d6e-67271aeca381 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:26.755470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-2dc72f69-a3e8-4169-8d6e-67271aeca381 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:26.755470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:26.755470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:26.755470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:26.755470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:26.755470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:26.755470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:26.755470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:26.755470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:26.755470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:26.755470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:26.755470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:26.755470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:00:26.755470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:00:26.755470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:00:26.755470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:00:26.756423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:26.756423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:26.756423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:26.756423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:26.756423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:26.756423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:26.756423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:26.756423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:26.756423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:26.756423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:26.756423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:26.756423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:26.756423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:26.756423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:26.756423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:26.756423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:26.756423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:00:26.757642 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:00:26.757642 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:26.757642 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:26.757642 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 401/817] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:26.762439 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf04652-7b71-48aa-a91b-cb06cb26313e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:26.814593 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc72f69-a3e8-4169-8d6e-67271aeca381 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:26.861777 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf04652-7b71-48aa-a91b-cb06cb26313e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:26.894999 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341183e-ba3d-48a1-b14f-5f9d73d1c2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:26.895820 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341183e-ba3d-48a1-b14f-5f9d73d1c2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:26.896098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341183e-ba3d-48a1-b14f-5f9d73d1c2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:26.896367 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341183e-ba3d-48a1-b14f-5f9d73d1c2d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:26.897005 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341183e-ba3d-48a1-b14f-5f9d73d1c2d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:26.897307 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341183e-ba3d-48a1-b14f-5f9d73d1c2d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:26.898690 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341183e-ba3d-48a1-b14f-5f9d73d1c2d6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:26.915298 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 417/818] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:26.916274 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341183e-ba3d-48a1-b14f-5f9d73d1c2d6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:26.917892 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc72f69-a3e8-4169-8d6e-67271aeca381 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:26.944311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:26.952512 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d152cfc-2296-4d69-bd90-cd1c0bbd5a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:26.953126 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d152cfc-2296-4d69-bd90-cd1c0bbd5a0b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:26.953407 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d152cfc-2296-4d69-bd90-cd1c0bbd5a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:26.953679 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d152cfc-2296-4d69-bd90-cd1c0bbd5a0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:26.962719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:26.964821 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d152cfc-2296-4d69-bd90-cd1c0bbd5a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:00:26.969543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-4d152cfc-2296-4d69-bd90-cd1c0bbd5a0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:00:26.969543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:26.969543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:26.969543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:26.969543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:26.969543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:26.969543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:26.969543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:26.969543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:26.969543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:26.969543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:26.969543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:26.969543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:00:26.969543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:00:26.969543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:00:26.969543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:00:26.970562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:00:26.970562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:00:26.970562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:00:26.970562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:00:26.970562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:26.970562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:26.970562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:26.970562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:26.970562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:26.970562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:26.970562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:26.970562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:26.970562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:26.970562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:26.970562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:26.970562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:26.971453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:26.971453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:26.971453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:26.971453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:26.971453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:00:26.971453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:00:26.971453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:00:26.971453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:00:26.971453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:00:26.971453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:00:26.971453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:00:26.971453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:26.971453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 402/819] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 03:00:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:00:26.976731 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d152cfc-2296-4d69-bd90-cd1c0bbd5a0b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:27.014872 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341183e-ba3d-48a1-b14f-5f9d73d1c2d6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:27.050830 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1999c0-77db-4145-8f9c-ec7e7aed06fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:27.051639 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1999c0-77db-4145-8f9c-ec7e7aed06fd None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:27.051748 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1999c0-77db-4145-8f9c-ec7e7aed06fd None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:27.052005 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1999c0-77db-4145-8f9c-ec7e7aed06fd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:27.059406 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1999c0-77db-4145-8f9c-ec7e7aed06fd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:27.060973 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1999c0-77db-4145-8f9c-ec7e7aed06fd None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:27.073100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-ec1999c0-77db-4145-8f9c-ec7e7aed06fd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 03:00:27.073100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:27.073100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:27.073100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:27.073100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:27.073100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:27.073100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:27.073100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:27.073100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:27.073100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:27.073100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:27.073100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:27.073100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:00:27.073100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:00:27.073100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:00:27.073100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:27.074069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:27.075073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:27.075073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:27.075073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:00:27.075073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:00:27.075073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:00:27.075073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:00:27.075073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:00:27.075073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:00:27.075073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 03:00:27.075073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:27.075073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 418/820] 10.222.0.27 () {56 vars in 1093 bytes} [Mon Oct 17 03:00:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:00:27.078452 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d152cfc-2296-4d69-bd90-cd1c0bbd5a0b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:27.079871 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1999c0-77db-4145-8f9c-ec7e7aed06fd None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:27.115207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec191d-6f76-4976-baba-6b34778d7d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:27.115817 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec191d-6f76-4976-baba-6b34778d7d7f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:27.116100 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec191d-6f76-4976-baba-6b34778d7d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:27.116376 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec191d-6f76-4976-baba-6b34778d7d7f None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:27.117039 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec191d-6f76-4976-baba-6b34778d7d7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:27.117343 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec191d-6f76-4976-baba-6b34778d7d7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:27.118722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec191d-6f76-4976-baba-6b34778d7d7f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:27.128202 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 403/821] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 03:00:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:27.134773 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec191d-6f76-4976-baba-6b34778d7d7f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:27.184714 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1999c0-77db-4145-8f9c-ec7e7aed06fd None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:27.184921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:27.191984 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:27.218450 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8c9b3-70ef-4a0d-acdd-03b517d19df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:27.219091 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8c9b3-70ef-4a0d-acdd-03b517d19df6 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:27.219391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8c9b3-70ef-4a0d-acdd-03b517d19df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:27.219652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8c9b3-70ef-4a0d-acdd-03b517d19df6 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:27.220378 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8c9b3-70ef-4a0d-acdd-03b517d19df6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:27.220684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8c9b3-70ef-4a0d-acdd-03b517d19df6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:27.222109 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8c9b3-70ef-4a0d-acdd-03b517d19df6 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:27.236596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec191d-6f76-4976-baba-6b34778d7d7f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:27.239555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 419/822] 10.222.0.27 () {56 vars in 1133 bytes} [Mon Oct 17 03:00:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:27.245193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8c9b3-70ef-4a0d-acdd-03b517d19df6 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:27.272010 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fad87-84af-41a0-b9ee-6effde886b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:27.272643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fad87-84af-41a0-b9ee-6effde886b65 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:27.272927 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fad87-84af-41a0-b9ee-6effde886b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:27.273202 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fad87-84af-41a0-b9ee-6effde886b65 None admin] PATH_INFO: `/v3/users/f5835258e02f4677a608dcd8e1416c96` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:27.289535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fad87-84af-41a0-b9ee-6effde886b65 None admin] RBAC: Authorizing `identity:get_user(user_id=f5835258e02f4677a608dcd8e1416c96)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:27.291033 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fad87-84af-41a0-b9ee-6effde886b65 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:27.292669 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 404/823] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 03:00:27 2022] GET /identity/v3/users/f5835258e02f4677a608dcd8e1416c96 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:27.298291 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fad87-84af-41a0-b9ee-6effde886b65 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:27.343311 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8c9b3-70ef-4a0d-acdd-03b517d19df6 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:27.378369 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4857cfc-5ded-4960-a9be-f3237d267ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:27.379138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4857cfc-5ded-4960-a9be-f3237d267ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:27.379417 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4857cfc-5ded-4960-a9be-f3237d267ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:27.379693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4857cfc-5ded-4960-a9be-f3237d267ea7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:27.395768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fad87-84af-41a0-b9ee-6effde886b65 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:27.404818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:27.409745 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:27.429967 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39355c-c8d5-41fc-9630-53b5e65509ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:27.430586 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39355c-c8d5-41fc-9630-53b5e65509ed None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:27.430908 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39355c-c8d5-41fc-9630-53b5e65509ed None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:27.431160 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39355c-c8d5-41fc-9630-53b5e65509ed None admin] PATH_INFO: `/v3/projects/6f73a6de7ebf4380bd8c884dc9c53f11` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:27.432276 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39355c-c8d5-41fc-9630-53b5e65509ed None admin] RBAC: Authorizing `identity:get_project(project_id=6f73a6de7ebf4380bd8c884dc9c53f11)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:27.433688 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39355c-c8d5-41fc-9630-53b5e65509ed None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:27.440233 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 405/824] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 03:00:27 2022] GET /identity/v3/projects/6f73a6de7ebf4380bd8c884dc9c53f11 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:27.441233 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39355c-c8d5-41fc-9630-53b5e65509ed None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:27.448346 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4857cfc-5ded-4960-a9be-f3237d267ea7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:27.449785 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4857cfc-5ded-4960-a9be-f3237d267ea7 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:27.538066 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39355c-c8d5-41fc-9630-53b5e65509ed None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:27.547927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 420/825] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 03:00:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 17 03:00:27.553935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4857cfc-5ded-4960-a9be-f3237d267ea7 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:27.571562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc875544-ec25-4f1b-b626-98f53f7799cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:27.572248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc875544-ec25-4f1b-b626-98f53f7799cc None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:27.572536 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc875544-ec25-4f1b-b626-98f53f7799cc None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:27.572811 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc875544-ec25-4f1b-b626-98f53f7799cc None admin] PATH_INFO: `/v3/projects/6f73a6de7ebf4380bd8c884dc9c53f11/users/f5835258e02f4677a608dcd8e1416c96/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:27.574729 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc875544-ec25-4f1b-b626-98f53f7799cc None admin] RBAC: Authorizing `identity:create_grant(project_id=6f73a6de7ebf4380bd8c884dc9c53f11, user_id=f5835258e02f4677a608dcd8e1416c96, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:27.576197 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc875544-ec25-4f1b-b626-98f53f7799cc None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:27.590274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 406/826] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 03:00:27 2022] PUT /identity/v3/projects/6f73a6de7ebf4380bd8c884dc9c53f11/users/f5835258e02f4677a608dcd8e1416c96/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:00:27.614511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:27.656232 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4857cfc-5ded-4960-a9be-f3237d267ea7 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:27.693855 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd0afd-5d22-47d9-9c4b-d12ee3924e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:27.694513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd0afd-5d22-47d9-9c4b-d12ee3924e0b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:27.694795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd0afd-5d22-47d9-9c4b-d12ee3924e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:27.695099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd0afd-5d22-47d9-9c4b-d12ee3924e0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:27.705877 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd0afd-5d22-47d9-9c4b-d12ee3924e0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:27.708230 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd0afd-5d22-47d9-9c4b-d12ee3924e0b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:27.724279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-37dd0afd-5d22-47d9-9c4b-d12ee3924e0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:27.724279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:27.724279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:27.724279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:27.724279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:27.724279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:27.724279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:27.724279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:27.724279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:27.724279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:27.724279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:27.724279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:27.724279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:00:27.724279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:00:27.724279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:00:27.724279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:00:27.725506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:27.725506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:27.725506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:27.725506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:27.725506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:27.725506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:27.725506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:27.725506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:27.725506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:27.725506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:27.725506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:27.725506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:27.725506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:27.725506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:27.725506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:27.725506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:27.725506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:00:27.726570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:00:27.726570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:00:27.726570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:27.726570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 421/827] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:00:27.731521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc875544-ec25-4f1b-b626-98f53f7799cc None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:27.770519 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:27.835162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc875544-ec25-4f1b-b626-98f53f7799cc None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:27.868337 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a463b1f-509e-46e4-ad16-668ea35772c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:27.868877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a463b1f-509e-46e4-ad16-668ea35772c2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:27.869160 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a463b1f-509e-46e4-ad16-668ea35772c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:27.869437 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a463b1f-509e-46e4-ad16-668ea35772c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:27.870083 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a463b1f-509e-46e4-ad16-668ea35772c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:27.870390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a463b1f-509e-46e4-ad16-668ea35772c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:27.871799 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a463b1f-509e-46e4-ad16-668ea35772c2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:27.883793 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 407/828] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:27.889509 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd0afd-5d22-47d9-9c4b-d12ee3924e0b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:27.987875 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd0afd-5d22-47d9-9c4b-d12ee3924e0b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:28.003335 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:28.022707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea70bf-844f-4679-a947-3fbbaee10054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:28.023373 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea70bf-844f-4679-a947-3fbbaee10054 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:28.023649 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea70bf-844f-4679-a947-3fbbaee10054 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:28.023918 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea70bf-844f-4679-a947-3fbbaee10054 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:28.030312 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea70bf-844f-4679-a947-3fbbaee10054 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:28.031766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea70bf-844f-4679-a947-3fbbaee10054 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:28.042730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-39ea70bf-844f-4679-a947-3fbbaee10054 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:00:28.042730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:28.042730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:28.042730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:28.042730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:28.042730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:28.042730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:28.042730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:28.042730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:28.042730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:28.042730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:28.042730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:28.042730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:00:28.042730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:00:28.042730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:00:28.043802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:00:28.043802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:28.043802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:28.043802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:28.043802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:28.043802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:28.043802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:28.043802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:28.043802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:28.043802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:28.043802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:28.043802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:28.043802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:28.043802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:28.043802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:28.043802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:28.043802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:28.049868 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:00:28.049868 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:00:28.049868 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:00:28.049868 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:00:28.049868 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:00:28.049868 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:00:28.049868 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:00:28.049868 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:28.049868 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 422/829] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 03:00:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:28.050450 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a463b1f-509e-46e4-ad16-668ea35772c2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:28.143720 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:28.149916 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a463b1f-509e-46e4-ad16-668ea35772c2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:28.182423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93fb09-d13f-4f0d-96fc-bb046de5dd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:28.183066 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93fb09-d13f-4f0d-96fc-bb046de5dd68 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:28.183348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93fb09-d13f-4f0d-96fc-bb046de5dd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:28.183621 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93fb09-d13f-4f0d-96fc-bb046de5dd68 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:28.184307 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93fb09-d13f-4f0d-96fc-bb046de5dd68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:28.184609 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93fb09-d13f-4f0d-96fc-bb046de5dd68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:28.186035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93fb09-d13f-4f0d-96fc-bb046de5dd68 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:28.198712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 408/830] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 03:00:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:00:28.204720 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea70bf-844f-4679-a947-3fbbaee10054 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:28.303594 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea70bf-844f-4679-a947-3fbbaee10054 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:28.336293 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c8bbe-d398-4d79-8ac8-f54bb44df6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:28.336891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c8bbe-d398-4d79-8ac8-f54bb44df6ee None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:28.337166 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c8bbe-d398-4d79-8ac8-f54bb44df6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:28.337433 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c8bbe-d398-4d79-8ac8-f54bb44df6ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:28.338145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c8bbe-d398-4d79-8ac8-f54bb44df6ee None admin] RBAC: Adding query filter params (role.id=355fbb07456348ac8f6ae42b3dd96690, scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2, user.id=01637fc07ba846aebcb42325bed46216) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:28.338451 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c8bbe-d398-4d79-8ac8-f54bb44df6ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:28.339892 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c8bbe-d398-4d79-8ac8-f54bb44df6ee None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:28.345433 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 423/831] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 03:00:28 2022] GET /identity/v3/role_assignments?user.id=01637fc07ba846aebcb42325bed46216&scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2&role.id=355fbb07456348ac8f6ae42b3dd96690 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:29.732587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7db0ee-8c60-449a-9bbe-f366b6a74990 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:29.733132 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7db0ee-8c60-449a-9bbe-f366b6a74990 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:29.733132 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7db0ee-8c60-449a-9bbe-f366b6a74990 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:29.734451 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 409/832] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:29.740244 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a98ae-9c53-4117-9ae5-2a61e41f4dfb None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:29.740660 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a98ae-9c53-4117-9ae5-2a61e41f4dfb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:29.740797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a98ae-9c53-4117-9ae5-2a61e41f4dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:29.759590 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-941a98ae-9c53-4117-9ae5-2a61e41f4dfb None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:29.760784 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-941a98ae-9c53-4117-9ae5-2a61e41f4dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:29.793175 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:29.825504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 424/833] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:30.092156 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7db0ee-8c60-449a-9bbe-f366b6a74990 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:30.306725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7db0ee-8c60-449a-9bbe-f366b6a74990 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:30.339216 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288dc2f-734c-4f4b-84d1-4220e15ba2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:30.339820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288dc2f-734c-4f4b-84d1-4220e15ba2cd None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:30.340092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288dc2f-734c-4f4b-84d1-4220e15ba2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:30.340359 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288dc2f-734c-4f4b-84d1-4220e15ba2cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:30.350166 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288dc2f-734c-4f4b-84d1-4220e15ba2cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:00:30.355099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-d288dc2f-734c-4f4b-84d1-4220e15ba2cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:00:30.355099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:30.355099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:30.355099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:30.355099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:30.355099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:30.355099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:30.355099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:30.355099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:30.355099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:30.355099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:30.355099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:30.355099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:00:30.355099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:00:30.355099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:00:30.355099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:00:30.356178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:00:30.356178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:00:30.356178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:00:30.356178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:00:30.356178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:30.356178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:30.356178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:30.356178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:30.356178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:30.356178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:30.356178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:30.356178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:30.356178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:30.356178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:30.356178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:30.356178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:30.357427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:30.357427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:30.357427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:30.357427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:30.357427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:00:30.357427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:00:30.357427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:00:30.357427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:00:30.357427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:00:30.357427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:00:30.357427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:00:30.357427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:30.357427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 410/834] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 03:00:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:30.362236 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a98ae-9c53-4117-9ae5-2a61e41f4dfb None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:30.458514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a98ae-9c53-4117-9ae5-2a61e41f4dfb None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:30.603812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799834d-ee87-41b2-aebf-18342fbc176c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:30.604328 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799834d-ee87-41b2-aebf-18342fbc176c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:30.604608 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799834d-ee87-41b2-aebf-18342fbc176c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:30.604880 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799834d-ee87-41b2-aebf-18342fbc176c None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:30.605537 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799834d-ee87-41b2-aebf-18342fbc176c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:30.605840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799834d-ee87-41b2-aebf-18342fbc176c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:30.607201 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799834d-ee87-41b2-aebf-18342fbc176c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:30.615401 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 425/835] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 03:00:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:30.621314 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288dc2f-734c-4f4b-84d1-4220e15ba2cd None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:30.639868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:30.717904 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288dc2f-734c-4f4b-84d1-4220e15ba2cd None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:30.749599 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fd8dc-a122-477c-9c41-dbd00dd71309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:30.750213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fd8dc-a122-477c-9c41-dbd00dd71309 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:30.750490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fd8dc-a122-477c-9c41-dbd00dd71309 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:30.750763 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fd8dc-a122-477c-9c41-dbd00dd71309 None admin] PATH_INFO: `/v3/users/f5835258e02f4677a608dcd8e1416c96` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:30.751900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38fd8dc-a122-477c-9c41-dbd00dd71309 None admin] RBAC: Authorizing `identity:get_user(user_id=f5835258e02f4677a608dcd8e1416c96)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:30.753351 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38fd8dc-a122-477c-9c41-dbd00dd71309 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:30.755447 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 411/836] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 03:00:30 2022] GET /identity/v3/users/f5835258e02f4677a608dcd8e1416c96 => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:30.761573 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799834d-ee87-41b2-aebf-18342fbc176c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:30.791946 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:30.856379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799834d-ee87-41b2-aebf-18342fbc176c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:30.889288 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c7719-d336-4c5b-94e7-273c7aff31a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:30.889886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c7719-d336-4c5b-94e7-273c7aff31a9 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:30.890165 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c7719-d336-4c5b-94e7-273c7aff31a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:30.890437 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c7719-d336-4c5b-94e7-273c7aff31a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:30.954074 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c7719-d336-4c5b-94e7-273c7aff31a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:30.955533 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c7719-d336-4c5b-94e7-273c7aff31a9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:30.983457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:31.047884 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 426/837] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 03:00:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 03:00:31.054492 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fd8dc-a122-477c-9c41-dbd00dd71309 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:31.135910 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:31.151140 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fd8dc-a122-477c-9c41-dbd00dd71309 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:31.183123 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a74ae-2577-43fb-a45a-2f63f93bb7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:31.183734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a74ae-2577-43fb-a45a-2f63f93bb7cb None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:31.184009 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a74ae-2577-43fb-a45a-2f63f93bb7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:31.184277 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a74ae-2577-43fb-a45a-2f63f93bb7cb None admin] PATH_INFO: `/v3/projects/6f73a6de7ebf4380bd8c884dc9c53f11` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:31.185352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a74ae-2577-43fb-a45a-2f63f93bb7cb None admin] RBAC: Authorizing `identity:get_project(project_id=6f73a6de7ebf4380bd8c884dc9c53f11)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:31.186768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a74ae-2577-43fb-a45a-2f63f93bb7cb None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:31.188388 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 412/838] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 03:00:31 2022] GET /identity/v3/projects/6f73a6de7ebf4380bd8c884dc9c53f11 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:31.195337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c7719-d336-4c5b-94e7-273c7aff31a9 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:31.290182 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c7719-d336-4c5b-94e7-273c7aff31a9 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:31.319336 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:31.325021 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134f49c-7b89-42bb-a44f-d02c3f5c66c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:31.325623 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134f49c-7b89-42bb-a44f-d02c3f5c66c8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:31.325900 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134f49c-7b89-42bb-a44f-d02c3f5c66c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:31.326170 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134f49c-7b89-42bb-a44f-d02c3f5c66c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:31.326911 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134f49c-7b89-42bb-a44f-d02c3f5c66c8 None admin] RBAC: Adding query filter params (role.id=ea0fd0c10f2f4e529270edbe3525d4c6, scope.project.id=6f73a6de7ebf4380bd8c884dc9c53f11, user.id=f5835258e02f4677a608dcd8e1416c96) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:31.327221 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134f49c-7b89-42bb-a44f-d02c3f5c66c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:31.328625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134f49c-7b89-42bb-a44f-d02c3f5c66c8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:31.334460 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 427/839] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 03:00:31 2022] GET /identity/v3/role_assignments?user.id=f5835258e02f4677a608dcd8e1416c96&scope.project.id=6f73a6de7ebf4380bd8c884dc9c53f11&role.id=ea0fd0c10f2f4e529270edbe3525d4c6 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:33.447741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87974c5b-d740-4f93-80fc-c8aae533d688 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:33.448257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87974c5b-d740-4f93-80fc-c8aae533d688 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:33.448343 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87974c5b-d740-4f93-80fc-c8aae533d688 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:33.449608 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 413/840] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:33.454747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fb5d0-77ab-4ffa-ac69-c67577aaabb8 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:33.455149 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fb5d0-77ab-4ffa-ac69-c67577aaabb8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:33.455326 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fb5d0-77ab-4ffa-ac69-c67577aaabb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:33.474194 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7b9fb5d0-77ab-4ffa-ac69-c67577aaabb8 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:33.475415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7b9fb5d0-77ab-4ffa-ac69-c67577aaabb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:33.539779 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 428/841] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:33.805392 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87974c5b-d740-4f93-80fc-c8aae533d688 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:33.904398 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87974c5b-d740-4f93-80fc-c8aae533d688 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:33.936019 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b71e86-46b9-452f-9b36-0ab656867508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:33.936640 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b71e86-46b9-452f-9b36-0ab656867508 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:33.936919 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b71e86-46b9-452f-9b36-0ab656867508 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:33.937190 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b71e86-46b9-452f-9b36-0ab656867508 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:33.938281 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b71e86-46b9-452f-9b36-0ab656867508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:33.939737 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b71e86-46b9-452f-9b36-0ab656867508 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:33.941821 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 414/842] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:33.947595 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fb5d0-77ab-4ffa-ac69-c67577aaabb8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:34.041737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fb5d0-77ab-4ffa-ac69-c67577aaabb8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:34.067651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:34.076508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2481c-e25b-4580-bdf4-3f79156415c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:34.077109 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2481c-e25b-4580-bdf4-3f79156415c9 None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:34.077390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2481c-e25b-4580-bdf4-3f79156415c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:34.077661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2481c-e25b-4580-bdf4-3f79156415c9 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:34.078312 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2481c-e25b-4580-bdf4-3f79156415c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:34.079756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2481c-e25b-4580-bdf4-3f79156415c9 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:34.101327 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 429/843] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 03:00:33 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:00:36.231481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c03d0-a432-44ce-b755-a64bce020424 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:36.232008 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c03d0-a432-44ce-b755-a64bce020424 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:36.232008 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c03d0-a432-44ce-b755-a64bce020424 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:36.233344 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 415/844] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:36.239041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866db2c-e842-4605-810c-00aeba9ce4fd None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:36.239456 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866db2c-e842-4605-810c-00aeba9ce4fd None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:36.239593 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866db2c-e842-4605-810c-00aeba9ce4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:36.257458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6866db2c-e842-4605-810c-00aeba9ce4fd None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:36.258708 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6866db2c-e842-4605-810c-00aeba9ce4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:36.321784 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 430/845] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:36.594855 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c03d0-a432-44ce-b755-a64bce020424 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:36.616427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:36.696199 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c03d0-a432-44ce-b755-a64bce020424 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:36.729892 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b06ac-42e1-4a03-9670-ff847fe2ac95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:36.730504 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b06ac-42e1-4a03-9670-ff847fe2ac95 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:36.730782 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b06ac-42e1-4a03-9670-ff847fe2ac95 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:36.731081 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b06ac-42e1-4a03-9670-ff847fe2ac95 None admin] PATH_INFO: `/v3/roles/baf207ce98e448f1a2e10c3310a0e33b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:36.732446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b06ac-42e1-4a03-9670-ff847fe2ac95 None admin] RBAC: Authorizing `identity:get_role(role_id=baf207ce98e448f1a2e10c3310a0e33b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:36.733825 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b06ac-42e1-4a03-9670-ff847fe2ac95 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:36.735145 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 416/846] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 03:00:36 2022] GET /identity/v3/roles/baf207ce98e448f1a2e10c3310a0e33b => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:36.740873 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866db2c-e842-4605-810c-00aeba9ce4fd None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:36.836739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866db2c-e842-4605-810c-00aeba9ce4fd None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:36.849206 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:36.870750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb02b1-feb7-4cd9-ac26-0619829f867e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:36.871406 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb02b1-feb7-4cd9-ac26-0619829f867e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:36.871686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb02b1-feb7-4cd9-ac26-0619829f867e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:36.871959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb02b1-feb7-4cd9-ac26-0619829f867e None admin] PATH_INFO: `/v3/users/bf48c70bb23e45a88b6b7d39540d3562` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:36.889019 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb02b1-feb7-4cd9-ac26-0619829f867e None admin] RBAC: Authorizing `identity:get_user(user_id=bf48c70bb23e45a88b6b7d39540d3562)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:36.890444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cb02b1-feb7-4cd9-ac26-0619829f867e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:36.892074 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 431/847] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 03:00:36 2022] GET /identity/v3/users/bf48c70bb23e45a88b6b7d39540d3562 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:36.898911 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b06ac-42e1-4a03-9670-ff847fe2ac95 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:36.983833 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:36.995920 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b06ac-42e1-4a03-9670-ff847fe2ac95 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:37.029798 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9421dd1-e680-4dce-8f13-3c9a2b8cd331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:37.030415 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9421dd1-e680-4dce-8f13-3c9a2b8cd331 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:37.030691 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9421dd1-e680-4dce-8f13-3c9a2b8cd331 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:37.030995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9421dd1-e680-4dce-8f13-3c9a2b8cd331 None admin] PATH_INFO: `/v3/projects/6f73a6de7ebf4380bd8c884dc9c53f11` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:37.032099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9421dd1-e680-4dce-8f13-3c9a2b8cd331 None admin] RBAC: Authorizing `identity:get_project(project_id=6f73a6de7ebf4380bd8c884dc9c53f11)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:37.033516 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9421dd1-e680-4dce-8f13-3c9a2b8cd331 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:37.035633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 417/848] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 03:00:36 2022] GET /identity/v3/projects/6f73a6de7ebf4380bd8c884dc9c53f11 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:37.041387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb02b1-feb7-4cd9-ac26-0619829f867e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:37.136683 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb02b1-feb7-4cd9-ac26-0619829f867e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:37.168619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcee336-22be-4d55-8a93-70a65838fca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:37.169263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcee336-22be-4d55-8a93-70a65838fca2 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:37.169539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcee336-22be-4d55-8a93-70a65838fca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:37.169810 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcee336-22be-4d55-8a93-70a65838fca2 None admin] PATH_INFO: `/v3/projects/6f73a6de7ebf4380bd8c884dc9c53f11/users/bf48c70bb23e45a88b6b7d39540d3562/roles/baf207ce98e448f1a2e10c3310a0e33b` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:37.171920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcee336-22be-4d55-8a93-70a65838fca2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f73a6de7ebf4380bd8c884dc9c53f11, user_id=bf48c70bb23e45a88b6b7d39540d3562, role_id=baf207ce98e448f1a2e10c3310a0e33b)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:37.173330 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcee336-22be-4d55-8a93-70a65838fca2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:37.186225 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 432/849] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 03:00:37 2022] PUT /identity/v3/projects/6f73a6de7ebf4380bd8c884dc9c53f11/users/bf48c70bb23e45a88b6b7d39540d3562/roles/baf207ce98e448f1a2e10c3310a0e33b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:00:39.276002 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686660d6-a804-47ba-96ff-a268795172f5 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:39.276521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686660d6-a804-47ba-96ff-a268795172f5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:39.276630 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686660d6-a804-47ba-96ff-a268795172f5 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:39.277934 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 418/850] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:39.285517 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e140c8-7c5a-49e0-8b30-2bdaed1aad00 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:39.285985 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e140c8-7c5a-49e0-8b30-2bdaed1aad00 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:39.286201 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e140c8-7c5a-49e0-8b30-2bdaed1aad00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:39.304147 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-12e140c8-7c5a-49e0-8b30-2bdaed1aad00 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:39.305306 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-12e140c8-7c5a-49e0-8b30-2bdaed1aad00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:39.327583 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:39.369402 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 433/851] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:39.637700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686660d6-a804-47ba-96ff-a268795172f5 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:39.734377 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686660d6-a804-47ba-96ff-a268795172f5 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:39.766982 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9230d85-e13f-430a-b683-33826b542e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:39.767601 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9230d85-e13f-430a-b683-33826b542e61 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:39.767881 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9230d85-e13f-430a-b683-33826b542e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:39.768154 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9230d85-e13f-430a-b683-33826b542e61 None admin] PATH_INFO: `/v3/roles/baf207ce98e448f1a2e10c3310a0e33b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:39.769623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9230d85-e13f-430a-b683-33826b542e61 None admin] RBAC: Authorizing `identity:get_role(role_id=baf207ce98e448f1a2e10c3310a0e33b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:39.771012 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9230d85-e13f-430a-b683-33826b542e61 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:39.772314 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 419/852] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 03:00:39 2022] GET /identity/v3/roles/baf207ce98e448f1a2e10c3310a0e33b => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:39.777918 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e140c8-7c5a-49e0-8b30-2bdaed1aad00 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:39.871267 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e140c8-7c5a-49e0-8b30-2bdaed1aad00 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:39.905263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc14d5-4715-4eab-a132-86c5fba6846d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:39.905826 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc14d5-4715-4eab-a132-86c5fba6846d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:39.906106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc14d5-4715-4eab-a132-86c5fba6846d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:39.906377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc14d5-4715-4eab-a132-86c5fba6846d None admin] PATH_INFO: `/v3/users/bf48c70bb23e45a88b6b7d39540d3562` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:39.907577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc14d5-4715-4eab-a132-86c5fba6846d None admin] RBAC: Authorizing `identity:get_user(user_id=bf48c70bb23e45a88b6b7d39540d3562)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:39.909000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc14d5-4715-4eab-a132-86c5fba6846d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:39.911090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 434/853] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 03:00:39 2022] GET /identity/v3/users/bf48c70bb23e45a88b6b7d39540d3562 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:39.916714 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9230d85-e13f-430a-b683-33826b542e61 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:39.941140 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:40.011857 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9230d85-e13f-430a-b683-33826b542e61 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:40.043548 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d03004-29d1-4316-b56e-691fac1bd905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:40.044158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d03004-29d1-4316-b56e-691fac1bd905 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:40.044434 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d03004-29d1-4316-b56e-691fac1bd905 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:40.044703 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d03004-29d1-4316-b56e-691fac1bd905 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:40.106682 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d03004-29d1-4316-b56e-691fac1bd905 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:40.108141 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d03004-29d1-4316-b56e-691fac1bd905 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:40.131063 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:40.202787 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 420/854] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 03:00:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 03:00:40.208588 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc14d5-4715-4eab-a132-86c5fba6846d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:40.235165 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:40.305131 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc14d5-4715-4eab-a132-86c5fba6846d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:40.337205 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffe661-23fa-4767-9bac-0e9c33e6fbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:40.337820 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffe661-23fa-4767-9bac-0e9c33e6fbc1 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:40.338097 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffe661-23fa-4767-9bac-0e9c33e6fbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:40.338369 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffe661-23fa-4767-9bac-0e9c33e6fbc1 None admin] PATH_INFO: `/v3/projects/6f73a6de7ebf4380bd8c884dc9c53f11` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:40.339469 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffe661-23fa-4767-9bac-0e9c33e6fbc1 None admin] RBAC: Authorizing `identity:get_project(project_id=6f73a6de7ebf4380bd8c884dc9c53f11)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:40.340873 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffe661-23fa-4767-9bac-0e9c33e6fbc1 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:40.342460 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 435/855] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 03:00:40 2022] GET /identity/v3/projects/6f73a6de7ebf4380bd8c884dc9c53f11 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:40.349402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d03004-29d1-4316-b56e-691fac1bd905 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:40.444753 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d03004-29d1-4316-b56e-691fac1bd905 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:40.469778 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:40.480658 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8195e-11e5-455f-97f4-fb5d52784564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:40.481258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8195e-11e5-455f-97f4-fb5d52784564 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:40.481542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8195e-11e5-455f-97f4-fb5d52784564 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:40.481818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8195e-11e5-455f-97f4-fb5d52784564 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:40.482537 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8195e-11e5-455f-97f4-fb5d52784564 None admin] RBAC: Adding query filter params (role.id=baf207ce98e448f1a2e10c3310a0e33b, scope.project.id=6f73a6de7ebf4380bd8c884dc9c53f11, user.id=bf48c70bb23e45a88b6b7d39540d3562) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:40.482850 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8195e-11e5-455f-97f4-fb5d52784564 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:40.484318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8195e-11e5-455f-97f4-fb5d52784564 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:40.489899 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 421/856] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 03:00:40 2022] GET /identity/v3/role_assignments?user.id=bf48c70bb23e45a88b6b7d39540d3562&scope.project.id=6f73a6de7ebf4380bd8c884dc9c53f11&role.id=baf207ce98e448f1a2e10c3310a0e33b => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:42.612809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1a949-5864-4f0d-a6a7-d524d4fac0d1 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:42.613893 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1a949-5864-4f0d-a6a7-d524d4fac0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:42.613995 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1a949-5864-4f0d-a6a7-d524d4fac0d1 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:42.615472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 436/857] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:42.620956 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d6b47-1f54-4ccd-a908-4ef2d90cda8b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:42.621378 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d6b47-1f54-4ccd-a908-4ef2d90cda8b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:42.621484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d6b47-1f54-4ccd-a908-4ef2d90cda8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:42.641265 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-be3d6b47-1f54-4ccd-a908-4ef2d90cda8b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:42.642439 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-be3d6b47-1f54-4ccd-a908-4ef2d90cda8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:42.706080 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 422/858] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:42.977773 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1a949-5864-4f0d-a6a7-d524d4fac0d1 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:43.069439 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:43.076169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1a949-5864-4f0d-a6a7-d524d4fac0d1 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:43.114264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2e95c-792c-4dda-baa4-86564dd42156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:43.114915 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2e95c-792c-4dda-baa4-86564dd42156 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:43.115186 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2e95c-792c-4dda-baa4-86564dd42156 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:43.115457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2e95c-792c-4dda-baa4-86564dd42156 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:43.116558 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2e95c-792c-4dda-baa4-86564dd42156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:43.117975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2e95c-792c-4dda-baa4-86564dd42156 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:43.119553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 437/859] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:43.125477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d6b47-1f54-4ccd-a908-4ef2d90cda8b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:43.220358 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d6b47-1f54-4ccd-a908-4ef2d90cda8b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:43.237396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:43.253562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba359e8-e636-4503-96d1-9da22694c397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:43.254197 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba359e8-e636-4503-96d1-9da22694c397 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:43.254472 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba359e8-e636-4503-96d1-9da22694c397 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:43.254739 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba359e8-e636-4503-96d1-9da22694c397 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:43.255420 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba359e8-e636-4503-96d1-9da22694c397 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:43.256834 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba359e8-e636-4503-96d1-9da22694c397 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:43.283714 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 423/860] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Oct 17 03:00:43 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:00:45.496181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff9332-d127-48f0-acf9-878ff3432058 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:45.496765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff9332-d127-48f0-acf9-878ff3432058 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:45.496765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff9332-d127-48f0-acf9-878ff3432058 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:45.498072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 438/861] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:45.503218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24d192-3a34-4124-a73e-275da3e3424e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:45.504362 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24d192-3a34-4124-a73e-275da3e3424e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:45.504872 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24d192-3a34-4124-a73e-275da3e3424e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:45.522266 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-0f24d192-3a34-4124-a73e-275da3e3424e None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:45.523465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0f24d192-3a34-4124-a73e-275da3e3424e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:45.586405 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 424/862] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:00:45.852489 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff9332-d127-48f0-acf9-878ff3432058 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:45.950735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff9332-d127-48f0-acf9-878ff3432058 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:45.983673 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29950d0-bd10-4fef-a6eb-40f451f3bd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:45.984287 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29950d0-bd10-4fef-a6eb-40f451f3bd0f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:45.984567 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29950d0-bd10-4fef-a6eb-40f451f3bd0f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:45.984840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29950d0-bd10-4fef-a6eb-40f451f3bd0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:45.985933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29950d0-bd10-4fef-a6eb-40f451f3bd0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:45.987404 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29950d0-bd10-4fef-a6eb-40f451f3bd0f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:45.989386 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 439/863] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:00:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:45.995065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24d192-3a34-4124-a73e-275da3e3424e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:46.085983 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:46.088935 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24d192-3a34-4124-a73e-275da3e3424e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:46.122810 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8ed18-7bc9-455b-a649-49710e47ccd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:46.123390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8ed18-7bc9-455b-a649-49710e47ccd2 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:46.123669 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8ed18-7bc9-455b-a649-49710e47ccd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:46.123939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8ed18-7bc9-455b-a649-49710e47ccd2 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:46.124583 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8ed18-7bc9-455b-a649-49710e47ccd2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:46.126001 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8ed18-7bc9-455b-a649-49710e47ccd2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:46.146026 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 425/864] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 03:00:45 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:00:48.296042 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b54377-27de-40ff-ab10-8746a987ed44 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:48.296528 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b54377-27de-40ff-ab10-8746a987ed44 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:48.296624 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b54377-27de-40ff-ab10-8746a987ed44 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:48.297912 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 440/865] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:48.303554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d82c4-a1e2-4c77-8d2b-f3c35e5837a1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:48.303956 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d82c4-a1e2-4c77-8d2b-f3c35e5837a1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:48.304095 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d82c4-a1e2-4c77-8d2b-f3c35e5837a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:48.321837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a14d82c4-a1e2-4c77-8d2b-f3c35e5837a1 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:48.323032 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a14d82c4-a1e2-4c77-8d2b-f3c35e5837a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:48.387367 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 426/866] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:48.662101 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b54377-27de-40ff-ab10-8746a987ed44 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:48.693828 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:48.762416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b54377-27de-40ff-ab10-8746a987ed44 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:48.795388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055b2dd-22f8-40bb-8b18-93e0d6239efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:48.796013 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055b2dd-22f8-40bb-8b18-93e0d6239efb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:48.796293 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055b2dd-22f8-40bb-8b18-93e0d6239efb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:48.796567 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055b2dd-22f8-40bb-8b18-93e0d6239efb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:48.806412 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055b2dd-22f8-40bb-8b18-93e0d6239efb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:00:48.811202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-c055b2dd-22f8-40bb-8b18-93e0d6239efb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:00:48.811202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:48.811202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:48.811202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:48.811202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:48.811202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:48.811202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:48.811202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:48.811202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:48.811202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:48.811202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:48.811202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:48.811202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:00:48.811202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:00:48.811202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:00:48.811202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:00:48.812366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:00:48.812366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:00:48.812366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:00:48.812366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:00:48.812366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:48.812366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:48.812366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:48.812366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:48.812366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:48.812366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:48.812366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:48.812366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:48.812366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:48.812366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:48.812366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:48.812366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:48.813500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:48.813500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:48.813500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:48.813500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:48.813500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:00:48.813500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:00:48.813500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:00:48.813500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:00:48.813500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:00:48.813500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:00:48.813500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:00:48.813500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:00:48.813500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 441/867] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 03:00:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:48.819153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d82c4-a1e2-4c77-8d2b-f3c35e5837a1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:48.901693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:48.917347 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d82c4-a1e2-4c77-8d2b-f3c35e5837a1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:48.950573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff2b43-3007-491c-9a75-abc958c71e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:48.951201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff2b43-3007-491c-9a75-abc958c71e63 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:48.951480 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff2b43-3007-491c-9a75-abc958c71e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:48.951752 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff2b43-3007-491c-9a75-abc958c71e63 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:48.952412 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff2b43-3007-491c-9a75-abc958c71e63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:48.952713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff2b43-3007-491c-9a75-abc958c71e63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:48.954072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff2b43-3007-491c-9a75-abc958c71e63 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:48.962809 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 427/868] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 03:00:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:48.969126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055b2dd-22f8-40bb-8b18-93e0d6239efb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:49.057290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:49.066954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055b2dd-22f8-40bb-8b18-93e0d6239efb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:49.099801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe8c13-0b3c-4085-abba-055985b8c283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:49.100419 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe8c13-0b3c-4085-abba-055985b8c283 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:49.100695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe8c13-0b3c-4085-abba-055985b8c283 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:49.100964 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe8c13-0b3c-4085-abba-055985b8c283 None admin] PATH_INFO: `/v3/users/d8aa4b88d25b425c81d69ece617413d0` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:49.113858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe8c13-0b3c-4085-abba-055985b8c283 None admin] RBAC: Authorizing `identity:get_user(user_id=d8aa4b88d25b425c81d69ece617413d0)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:49.115309 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe8c13-0b3c-4085-abba-055985b8c283 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:49.116870 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 442/869] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 03:00:48 2022] GET /identity/v3/users/d8aa4b88d25b425c81d69ece617413d0 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:49.122509 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff2b43-3007-491c-9a75-abc958c71e63 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:49.224357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff2b43-3007-491c-9a75-abc958c71e63 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:49.256861 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ae5a4-e4e7-4c96-981d-6548029e1526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:49.257793 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ae5a4-e4e7-4c96-981d-6548029e1526 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:49.258075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ae5a4-e4e7-4c96-981d-6548029e1526 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:49.258345 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ae5a4-e4e7-4c96-981d-6548029e1526 None admin] PATH_INFO: `/v3/projects/d2b6590bfe1f4505be3453a518e791ef` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:49.259481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ae5a4-e4e7-4c96-981d-6548029e1526 None admin] RBAC: Authorizing `identity:get_project(project_id=d2b6590bfe1f4505be3453a518e791ef)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:49.260921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ae5a4-e4e7-4c96-981d-6548029e1526 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:49.262540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 428/870] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 03:00:49 2022] GET /identity/v3/projects/d2b6590bfe1f4505be3453a518e791ef => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:49.268487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe8c13-0b3c-4085-abba-055985b8c283 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:49.363813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe8c13-0b3c-4085-abba-055985b8c283 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:49.396115 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20407a8a-c7e5-489c-84dd-64b632512031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:49.396759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20407a8a-c7e5-489c-84dd-64b632512031 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:49.397061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20407a8a-c7e5-489c-84dd-64b632512031 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:49.397331 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20407a8a-c7e5-489c-84dd-64b632512031 None admin] PATH_INFO: `/v3/projects/d2b6590bfe1f4505be3453a518e791ef/users/d8aa4b88d25b425c81d69ece617413d0/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:49.399197 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20407a8a-c7e5-489c-84dd-64b632512031 None admin] RBAC: Authorizing `identity:create_grant(project_id=d2b6590bfe1f4505be3453a518e791ef, user_id=d8aa4b88d25b425c81d69ece617413d0, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:49.400589 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20407a8a-c7e5-489c-84dd-64b632512031 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:49.414427 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 443/871] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 03:00:49 2022] PUT /identity/v3/projects/d2b6590bfe1f4505be3453a518e791ef/users/d8aa4b88d25b425c81d69ece617413d0/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:00:51.540113 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476aeda8-58d5-432e-9373-6edb82f03e29 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:51.540627 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476aeda8-58d5-432e-9373-6edb82f03e29 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:51.540627 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476aeda8-58d5-432e-9373-6edb82f03e29 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:51.541905 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 429/872] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:51.547157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f0627-58e9-4845-9c5e-c3c1f648c4f5 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:51.547517 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f0627-58e9-4845-9c5e-c3c1f648c4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:51.547724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f0627-58e9-4845-9c5e-c3c1f648c4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:51.565864 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-972f0627-58e9-4845-9c5e-c3c1f648c4f5 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:51.567760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-972f0627-58e9-4845-9c5e-c3c1f648c4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:51.570394 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:51.631380 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 444/873] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:51.898816 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476aeda8-58d5-432e-9373-6edb82f03e29 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:51.917212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:51.996733 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476aeda8-58d5-432e-9373-6edb82f03e29 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:52.028186 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c353287-7b2c-44ab-81d2-9f8283d291b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:52.028795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c353287-7b2c-44ab-81d2-9f8283d291b6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:52.029070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c353287-7b2c-44ab-81d2-9f8283d291b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:52.029338 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c353287-7b2c-44ab-81d2-9f8283d291b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:52.040264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c353287-7b2c-44ab-81d2-9f8283d291b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:00:52.044901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-5c353287-7b2c-44ab-81d2-9f8283d291b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:00:52.044901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:00:52.044901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:00:52.044901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:00:52.044901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:00:52.044901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:00:52.044901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:00:52.044901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:00:52.044901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:00:52.044901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:00:52.044901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:00:52.044901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:00:52.044901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:00:52.044901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:00:52.044901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:00:52.044901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:00:52.045989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:00:52.045989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:00:52.045989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:00:52.045989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:00:52.045989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:00:52.045989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:00:52.045989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:00:52.045989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:00:52.045989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:00:52.045989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:00:52.045989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:00:52.045989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:00:52.045989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:00:52.045989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:00:52.045989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:00:52.045989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:00:52.047201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:00:52.047201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:00:52.047201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:00:52.047201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:00:52.047201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:00:52.047201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:00:52.047201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:00:52.047201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:00:52.047201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:00:52.047201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:00:52.047201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:00:52.047201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:00:52.047201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 430/874] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 03:00:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:00:52.052450 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f0627-58e9-4845-9c5e-c3c1f648c4f5 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:52.146263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f0627-58e9-4845-9c5e-c3c1f648c4f5 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:52.178816 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cccf5c-48b2-4158-9ea4-22f87ee32a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:52.179384 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cccf5c-48b2-4158-9ea4-22f87ee32a81 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:52.179660 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cccf5c-48b2-4158-9ea4-22f87ee32a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:52.179930 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cccf5c-48b2-4158-9ea4-22f87ee32a81 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:52.180582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cccf5c-48b2-4158-9ea4-22f87ee32a81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:52.180882 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cccf5c-48b2-4158-9ea4-22f87ee32a81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:52.182219 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cccf5c-48b2-4158-9ea4-22f87ee32a81 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:52.189784 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 445/875] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 03:00:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:52.195883 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c353287-7b2c-44ab-81d2-9f8283d291b6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:52.270198 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:52.293121 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c353287-7b2c-44ab-81d2-9f8283d291b6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:52.325592 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554334b-5e66-4303-8300-6e524b704838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:52.326207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554334b-5e66-4303-8300-6e524b704838 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:52.326487 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554334b-5e66-4303-8300-6e524b704838 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:52.326761 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554334b-5e66-4303-8300-6e524b704838 None admin] PATH_INFO: `/v3/users/d8aa4b88d25b425c81d69ece617413d0` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:52.327907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554334b-5e66-4303-8300-6e524b704838 None admin] RBAC: Authorizing `identity:get_user(user_id=d8aa4b88d25b425c81d69ece617413d0)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:52.329331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a554334b-5e66-4303-8300-6e524b704838 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:52.330903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 431/876] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 03:00:52 2022] GET /identity/v3/users/d8aa4b88d25b425c81d69ece617413d0 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:52.337567 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cccf5c-48b2-4158-9ea4-22f87ee32a81 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:52.340442 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:52.436500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cccf5c-48b2-4158-9ea4-22f87ee32a81 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:52.468965 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87007a-ba71-4b92-9a5c-0c5dbf5e1a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:52.469555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87007a-ba71-4b92-9a5c-0c5dbf5e1a6f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:52.469834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87007a-ba71-4b92-9a5c-0c5dbf5e1a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:52.470108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87007a-ba71-4b92-9a5c-0c5dbf5e1a6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:52.535963 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87007a-ba71-4b92-9a5c-0c5dbf5e1a6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:52.537399 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87007a-ba71-4b92-9a5c-0c5dbf5e1a6f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:52.539118 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:52.633508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 446/877] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 03:00:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 03:00:52.639611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554334b-5e66-4303-8300-6e524b704838 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:52.736516 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554334b-5e66-4303-8300-6e524b704838 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:52.768723 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed448cb1-1caa-4bec-8263-b69fd9759784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:52.769368 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed448cb1-1caa-4bec-8263-b69fd9759784 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:52.769650 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed448cb1-1caa-4bec-8263-b69fd9759784 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:52.769924 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed448cb1-1caa-4bec-8263-b69fd9759784 None admin] PATH_INFO: `/v3/projects/d2b6590bfe1f4505be3453a518e791ef` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:52.771123 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed448cb1-1caa-4bec-8263-b69fd9759784 None admin] RBAC: Authorizing `identity:get_project(project_id=d2b6590bfe1f4505be3453a518e791ef)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:52.772544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed448cb1-1caa-4bec-8263-b69fd9759784 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:52.774136 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 432/878] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 03:00:52 2022] GET /identity/v3/projects/d2b6590bfe1f4505be3453a518e791ef => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:52.780046 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87007a-ba71-4b92-9a5c-0c5dbf5e1a6f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:52.874251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87007a-ba71-4b92-9a5c-0c5dbf5e1a6f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:52.875733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:52.908283 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14cbd0-b93b-4c4b-8955-0b724e34e893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:52.908873 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14cbd0-b93b-4c4b-8955-0b724e34e893 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:52.909146 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14cbd0-b93b-4c4b-8955-0b724e34e893 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:52.909413 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14cbd0-b93b-4c4b-8955-0b724e34e893 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:52.910700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14cbd0-b93b-4c4b-8955-0b724e34e893 None admin] RBAC: Adding query filter params (role.id=ea0fd0c10f2f4e529270edbe3525d4c6, scope.project.id=d2b6590bfe1f4505be3453a518e791ef, user.id=d8aa4b88d25b425c81d69ece617413d0) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:00:52.911035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14cbd0-b93b-4c4b-8955-0b724e34e893 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:52.912426 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14cbd0-b93b-4c4b-8955-0b724e34e893 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:52.917884 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 447/879] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 03:00:52 2022] GET /identity/v3/role_assignments?user.id=d8aa4b88d25b425c81d69ece617413d0&scope.project.id=d2b6590bfe1f4505be3453a518e791ef&role.id=ea0fd0c10f2f4e529270edbe3525d4c6 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:55.062182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bbd33-9a9a-4e3e-b352-11f2a2295fa2 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:55.062769 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bbd33-9a9a-4e3e-b352-11f2a2295fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:55.062769 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bbd33-9a9a-4e3e-b352-11f2a2295fa2 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:55.064212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 433/880] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:55.069482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a85f5-86ac-4025-8a14-7202054bfba5 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:55.069892 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a85f5-86ac-4025-8a14-7202054bfba5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:55.070018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a85f5-86ac-4025-8a14-7202054bfba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:55.088026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-155a85f5-86ac-4025-8a14-7202054bfba5 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:55.089195 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-155a85f5-86ac-4025-8a14-7202054bfba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:55.152758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 448/881] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:55.417999 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bbd33-9a9a-4e3e-b352-11f2a2295fa2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:55.433133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:55.517777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bbd33-9a9a-4e3e-b352-11f2a2295fa2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:55.549968 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109a209-8681-43d7-8ab0-7cb4390ccace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:55.550598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109a209-8681-43d7-8ab0-7cb4390ccace None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:55.551131 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109a209-8681-43d7-8ab0-7cb4390ccace None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:55.551435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109a209-8681-43d7-8ab0-7cb4390ccace None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:55.552091 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109a209-8681-43d7-8ab0-7cb4390ccace None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:55.553445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109a209-8681-43d7-8ab0-7cb4390ccace None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:55.569224 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 434/882] 10.222.0.27 () {60 vars in 1138 bytes} [Mon Oct 17 03:00:55 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:00:57.755696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33074968-fd58-41b4-83fc-6213ba4483c6 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:57.756230 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33074968-fd58-41b4-83fc-6213ba4483c6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:57.756230 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33074968-fd58-41b4-83fc-6213ba4483c6 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:57.757582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 449/883] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:00:57.763059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c11518-5214-4040-871d-d16c99069f8b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:57.763491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c11518-5214-4040-871d-d16c99069f8b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:57.763591 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c11518-5214-4040-871d-d16c99069f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:57.781817 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-34c11518-5214-4040-871d-d16c99069f8b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:00:57.783008 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-34c11518-5214-4040-871d-d16c99069f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:00:57.848371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 435/884] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:00:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:00:58.117684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33074968-fd58-41b4-83fc-6213ba4483c6 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:58.209292 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:58.216353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33074968-fd58-41b4-83fc-6213ba4483c6 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:58.250041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb6e04-6972-4835-9383-02671e4c2b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:58.250658 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb6e04-6972-4835-9383-02671e4c2b17 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:58.250965 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb6e04-6972-4835-9383-02671e4c2b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:58.251237 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb6e04-6972-4835-9383-02671e4c2b17 None admin] PATH_INFO: `/v3/domains/021b5637974b45faac2dc37e2870497b` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:58.262357 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb6e04-6972-4835-9383-02671e4c2b17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=021b5637974b45faac2dc37e2870497b)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:58.263807 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb6e04-6972-4835-9383-02671e4c2b17 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:58.265368 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 450/885] 10.222.0.27 () {56 vars in 1183 bytes} [Mon Oct 17 03:00:58 2022] GET /identity/v3/domains/021b5637974b45faac2dc37e2870497b => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:00:58.271199 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c11518-5214-4040-871d-d16c99069f8b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:00:58.273335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:00:58.366028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c11518-5214-4040-871d-d16c99069f8b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:00:58.398669 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc027378-4b41-41a0-bc66-7cf3d3899f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:00:58.399248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc027378-4b41-41a0-bc66-7cf3d3899f10 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:00:58.399528 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc027378-4b41-41a0-bc66-7cf3d3899f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:00:58.399792 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc027378-4b41-41a0-bc66-7cf3d3899f10 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:00:58.400440 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc027378-4b41-41a0-bc66-7cf3d3899f10 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:00:58.401857 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc027378-4b41-41a0-bc66-7cf3d3899f10 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:00:58.428248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 436/886] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Oct 17 03:00:58 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:01:00.567131 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45f7b4-81ac-4d3e-a371-197c7aa1b7fa None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:00.567667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45f7b4-81ac-4d3e-a371-197c7aa1b7fa None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:00.567667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45f7b4-81ac-4d3e-a371-197c7aa1b7fa None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:00.569004 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 451/887] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:01:00.574267 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41edc6-dbc6-4648-9bc7-d9b55f6f7466 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:00.574678 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41edc6-dbc6-4648-9bc7-d9b55f6f7466 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:00.574813 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc41edc6-dbc6-4648-9bc7-d9b55f6f7466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:00.592994 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-dc41edc6-dbc6-4648-9bc7-d9b55f6f7466 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:01:00.594156 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-dc41edc6-dbc6-4648-9bc7-d9b55f6f7466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:01:00.640927 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:01:00.658351 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 437/888] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:01:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:01:00.926938 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45f7b4-81ac-4d3e-a371-197c7aa1b7fa None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:01:01.024621 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45f7b4-81ac-4d3e-a371-197c7aa1b7fa None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:01:01.056362 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc19f70-536e-41d0-82dc-f238dd33ba88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:01:01.056979 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc19f70-536e-41d0-82dc-f238dd33ba88 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:01.057258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc19f70-536e-41d0-82dc-f238dd33ba88 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:01.057531 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc19f70-536e-41d0-82dc-f238dd33ba88 None admin] PATH_INFO: `/v3/domains/021b5637974b45faac2dc37e2870497b` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:01.058626 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc19f70-536e-41d0-82dc-f238dd33ba88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=021b5637974b45faac2dc37e2870497b)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:01:01.060067 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc19f70-536e-41d0-82dc-f238dd33ba88 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:01:01.061592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 452/889] 10.222.0.27 () {56 vars in 1183 bytes} [Mon Oct 17 03:01:00 2022] GET /identity/v3/domains/021b5637974b45faac2dc37e2870497b => generated 267 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:01:01.067870 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41edc6-dbc6-4648-9bc7-d9b55f6f7466 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:01:01.163325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc41edc6-dbc6-4648-9bc7-d9b55f6f7466 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:01:01.196087 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d0d8b-c2b0-47fb-b609-1c2ad129b04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:01:01.196653 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d0d8b-c2b0-47fb-b609-1c2ad129b04c None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:01.197141 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d0d8b-c2b0-47fb-b609-1c2ad129b04c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:01.197536 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d0d8b-c2b0-47fb-b609-1c2ad129b04c None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:01.198204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d0d8b-c2b0-47fb-b609-1c2ad129b04c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:01:01.199672 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d0d8b-c2b0-47fb-b609-1c2ad129b04c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:01:01.222268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 438/890] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 03:01:01 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:01:03.384933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ad500-4fee-4f9d-bae6-20af5a55dee3 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:03.385478 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ad500-4fee-4f9d-bae6-20af5a55dee3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:03.385478 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ad500-4fee-4f9d-bae6-20af5a55dee3 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:03.386812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 453/891] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:01:03.391980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521b932-c970-422a-abf3-17ceda1379e1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:03.392358 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521b932-c970-422a-abf3-17ceda1379e1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:03.392580 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521b932-c970-422a-abf3-17ceda1379e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:03.411223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3521b932-c970-422a-abf3-17ceda1379e1 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:01:03.412407 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3521b932-c970-422a-abf3-17ceda1379e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:01:03.446270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:01:03.479831 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 439/892] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:01:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:01:03.748087 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ad500-4fee-4f9d-bae6-20af5a55dee3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:01:03.774385 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:01:03.849341 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ad500-4fee-4f9d-bae6-20af5a55dee3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:01:03.883694 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb13b7c-f086-42cc-a1f7-8ec5e6a5537e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:01:03.884548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb13b7c-f086-42cc-a1f7-8ec5e6a5537e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:03.884885 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb13b7c-f086-42cc-a1f7-8ec5e6a5537e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:03.885226 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb13b7c-f086-42cc-a1f7-8ec5e6a5537e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:03.897603 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb13b7c-f086-42cc-a1f7-8ec5e6a5537e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:01:03.902314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-0eb13b7c-f086-42cc-a1f7-8ec5e6a5537e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:01:03.902314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:01:03.902314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:01:03.902314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:01:03.902314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:01:03.902314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:01:03.902314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:01:03.902314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:01:03.902314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:01:03.902314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:01:03.902314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:01:03.902314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:01:03.902314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:01:03.902314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:01:03.902314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:01:03.902314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:01:03.903542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:01:03.903542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:01:03.903542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:01:03.903542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:01:03.903542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:01:03.903542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:01:03.903542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:01:03.903542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:01:03.903542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:01:03.903542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:01:03.903542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:01:03.903542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:01:03.903542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:01:03.903542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:01:03.903542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:01:03.903542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:01:03.904651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:01:03.904651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:01:03.904651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:01:03.904651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:01:03.904651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:01:03.904651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:01:03.904651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:01:03.904651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:01:03.904651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:01:03.904651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:01:03.904651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:01:03.904651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:01:03.904651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 454/893] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 03:01:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:01:03.909649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521b932-c970-422a-abf3-17ceda1379e1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:01:04.002190 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521b932-c970-422a-abf3-17ceda1379e1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:01:04.035162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d6c45-644d-4a7e-beb3-1e72a75ebedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:01:04.035752 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d6c45-644d-4a7e-beb3-1e72a75ebedd None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:04.036028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d6c45-644d-4a7e-beb3-1e72a75ebedd None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:04.036298 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d6c45-644d-4a7e-beb3-1e72a75ebedd None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:04.036951 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d6c45-644d-4a7e-beb3-1e72a75ebedd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:01:04.037249 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d6c45-644d-4a7e-beb3-1e72a75ebedd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:01:04.038600 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d6c45-644d-4a7e-beb3-1e72a75ebedd None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:01:04.046193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 440/894] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 03:01:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:01:04.052390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb13b7c-f086-42cc-a1f7-8ec5e6a5537e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:01:04.132554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:01:04.148339 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb13b7c-f086-42cc-a1f7-8ec5e6a5537e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:01:04.180502 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd0874-5c55-47ae-9d58-e1c07a384f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:01:04.181119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd0874-5c55-47ae-9d58-e1c07a384f1b None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:04.181404 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd0874-5c55-47ae-9d58-e1c07a384f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:04.181681 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd0874-5c55-47ae-9d58-e1c07a384f1b None admin] PATH_INFO: `/v3/users/7fa8a3b6ef4840a2b9ad4bdfddf7c615` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:04.195752 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd0874-5c55-47ae-9d58-e1c07a384f1b None admin] RBAC: Authorizing `identity:get_user(user_id=7fa8a3b6ef4840a2b9ad4bdfddf7c615)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:01:04.197184 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd0874-5c55-47ae-9d58-e1c07a384f1b None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:01:04.198747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 455/895] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 03:01:04 2022] GET /identity/v3/users/7fa8a3b6ef4840a2b9ad4bdfddf7c615 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:01:04.204345 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d6c45-644d-4a7e-beb3-1e72a75ebedd None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:01:04.234346 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:01:04.301402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d6c45-644d-4a7e-beb3-1e72a75ebedd None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:01:04.333373 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b020b1-d633-447e-b59c-833cc8d4b3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:01:04.333985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b020b1-d633-447e-b59c-833cc8d4b3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:04.334260 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b020b1-d633-447e-b59c-833cc8d4b3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:04.334527 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b020b1-d633-447e-b59c-833cc8d4b3b6 None admin] PATH_INFO: `/v3/projects/6f4bcb81158c45cba0f1e51a2eb84361` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:04.335647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b020b1-d633-447e-b59c-833cc8d4b3b6 None admin] RBAC: Authorizing `identity:get_project(project_id=6f4bcb81158c45cba0f1e51a2eb84361)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:01:04.337069 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b020b1-d633-447e-b59c-833cc8d4b3b6 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:01:04.338650 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 441/896] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 03:01:04 2022] GET /identity/v3/projects/6f4bcb81158c45cba0f1e51a2eb84361 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:01:04.344551 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd0874-5c55-47ae-9d58-e1c07a384f1b None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:01:04.440393 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd0874-5c55-47ae-9d58-e1c07a384f1b None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:01:04.473901 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1adc7c2-96ec-4648-9e60-8acfee881844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:01:04.474538 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1adc7c2-96ec-4648-9e60-8acfee881844 None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:04.474819 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1adc7c2-96ec-4648-9e60-8acfee881844 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:04.475121 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1adc7c2-96ec-4648-9e60-8acfee881844 None admin] PATH_INFO: `/v3/projects/6f4bcb81158c45cba0f1e51a2eb84361/users/7fa8a3b6ef4840a2b9ad4bdfddf7c615/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:04.477001 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1adc7c2-96ec-4648-9e60-8acfee881844 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f4bcb81158c45cba0f1e51a2eb84361, user_id=7fa8a3b6ef4840a2b9ad4bdfddf7c615, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:01:04.478419 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1adc7c2-96ec-4648-9e60-8acfee881844 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:01:04.491304 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 456/897] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 03:01:04 2022] PUT /identity/v3/projects/6f4bcb81158c45cba0f1e51a2eb84361/users/7fa8a3b6ef4840a2b9ad4bdfddf7c615/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:01:06.625642 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a31345-f17f-418a-82ea-18e8aec3ce5f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:06.626164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a31345-f17f-418a-82ea-18e8aec3ce5f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:06.626164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a31345-f17f-418a-82ea-18e8aec3ce5f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:06.627554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 442/898] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:01:06.633429 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217a678-d53d-4b81-9baa-2ddcb11493bd None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:06.633857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217a678-d53d-4b81-9baa-2ddcb11493bd None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:06.633955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217a678-d53d-4b81-9baa-2ddcb11493bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:06.640132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:01:06.653601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f217a678-d53d-4b81-9baa-2ddcb11493bd None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:01:06.654778 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f217a678-d53d-4b81-9baa-2ddcb11493bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:01:06.718929 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 457/899] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:01:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:01:06.983603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a31345-f17f-418a-82ea-18e8aec3ce5f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:01:07.077837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:01:07.082292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a31345-f17f-418a-82ea-18e8aec3ce5f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:01:07.114769 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322cc9a-9d03-46ae-ad81-952a2425e957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:01:07.115414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322cc9a-9d03-46ae-ad81-952a2425e957 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:07.115692 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322cc9a-9d03-46ae-ad81-952a2425e957 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:07.115960 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322cc9a-9d03-46ae-ad81-952a2425e957 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:07.126077 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322cc9a-9d03-46ae-ad81-952a2425e957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:01:07.130959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-5322cc9a-9d03-46ae-ad81-952a2425e957 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:01:07.130959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:01:07.130959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:01:07.130959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:01:07.130959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:01:07.130959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:01:07.130959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:01:07.130959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:01:07.130959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:01:07.130959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:01:07.130959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:01:07.130959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:01:07.130959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:01:07.130959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:01:07.130959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:01:07.130959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:01:07.132051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:01:07.132051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:01:07.132051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:01:07.132051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:01:07.132051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:01:07.132051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:01:07.132051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:01:07.132051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:01:07.132051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:01:07.132051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:01:07.132051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:01:07.132051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:01:07.132051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:01:07.132051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:01:07.132051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:01:07.132051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:01:07.132922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:01:07.132922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:01:07.132922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:01:07.132922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:01:07.132922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:01:07.132922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:01:07.132922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:01:07.132922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:01:07.132922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:01:07.132922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:01:07.132922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:01:07.132922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:01:07.132922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 443/900] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 03:01:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:01:07.138224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217a678-d53d-4b81-9baa-2ddcb11493bd None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:01:07.233329 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217a678-d53d-4b81-9baa-2ddcb11493bd None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:01:07.265703 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:01:07.271576 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4528bc8-bd2e-4a3c-926d-50d5bc7386cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:01:07.272184 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4528bc8-bd2e-4a3c-926d-50d5bc7386cb None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:07.272466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4528bc8-bd2e-4a3c-926d-50d5bc7386cb None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:07.272737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4528bc8-bd2e-4a3c-926d-50d5bc7386cb None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:07.273396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4528bc8-bd2e-4a3c-926d-50d5bc7386cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:01:07.273698 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4528bc8-bd2e-4a3c-926d-50d5bc7386cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:01:07.275105 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4528bc8-bd2e-4a3c-926d-50d5bc7386cb None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:01:07.283116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 458/901] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 03:01:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:01:07.289705 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322cc9a-9d03-46ae-ad81-952a2425e957 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:01:07.390497 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322cc9a-9d03-46ae-ad81-952a2425e957 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:01:07.423663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342e283-3fd9-47f4-be2a-f5c00fbe4439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:01:07.424280 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342e283-3fd9-47f4-be2a-f5c00fbe4439 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:07.424559 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342e283-3fd9-47f4-be2a-f5c00fbe4439 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:07.424831 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342e283-3fd9-47f4-be2a-f5c00fbe4439 None admin] PATH_INFO: `/v3/users/7fa8a3b6ef4840a2b9ad4bdfddf7c615` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:07.426185 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342e283-3fd9-47f4-be2a-f5c00fbe4439 None admin] RBAC: Authorizing `identity:get_user(user_id=7fa8a3b6ef4840a2b9ad4bdfddf7c615)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:01:07.427649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342e283-3fd9-47f4-be2a-f5c00fbe4439 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:01:07.429203 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 444/902] 10.222.0.27 () {56 vars in 1177 bytes} [Mon Oct 17 03:01:07 2022] GET /identity/v3/users/7fa8a3b6ef4840a2b9ad4bdfddf7c615 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:01:07.435163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4528bc8-bd2e-4a3c-926d-50d5bc7386cb None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:01:07.530984 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4528bc8-bd2e-4a3c-926d-50d5bc7386cb None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:01:07.562544 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4ab99-bd5d-436f-a7ca-f4d0a363fc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:01:07.563174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4ab99-bd5d-436f-a7ca-f4d0a363fc46 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:07.563456 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4ab99-bd5d-436f-a7ca-f4d0a363fc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:07.563729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4ab99-bd5d-436f-a7ca-f4d0a363fc46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:07.620614 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:01:07.627994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4ab99-bd5d-436f-a7ca-f4d0a363fc46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:01:07.629933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4ab99-bd5d-436f-a7ca-f4d0a363fc46 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:01:07.723217 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 459/903] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 03:01:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 17 03:01:07.729080 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342e283-3fd9-47f4-be2a-f5c00fbe4439 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:01:07.758251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:01:07.827558 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342e283-3fd9-47f4-be2a-f5c00fbe4439 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:01:07.858831 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee99d4e-3621-40fe-ad00-20fdd56c08f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:01:07.859474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee99d4e-3621-40fe-ad00-20fdd56c08f9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:07.859753 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee99d4e-3621-40fe-ad00-20fdd56c08f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:07.860021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee99d4e-3621-40fe-ad00-20fdd56c08f9 None admin] PATH_INFO: `/v3/projects/6f4bcb81158c45cba0f1e51a2eb84361` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:07.861109 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee99d4e-3621-40fe-ad00-20fdd56c08f9 None admin] RBAC: Authorizing `identity:get_project(project_id=6f4bcb81158c45cba0f1e51a2eb84361)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:01:07.862536 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee99d4e-3621-40fe-ad00-20fdd56c08f9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:01:07.864769 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 445/904] 10.222.0.27 () {56 vars in 1186 bytes} [Mon Oct 17 03:01:07 2022] GET /identity/v3/projects/6f4bcb81158c45cba0f1e51a2eb84361 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:01:07.870999 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4ab99-bd5d-436f-a7ca-f4d0a363fc46 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:01:07.965619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:01:07.972841 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4ab99-bd5d-436f-a7ca-f4d0a363fc46 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:01:08.007060 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee46a9-3051-4f31-9297-c9f92fb03b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:01:08.007699 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee46a9-3051-4f31-9297-c9f92fb03b46 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:01:08.008249 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee46a9-3051-4f31-9297-c9f92fb03b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:01:08.008546 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee46a9-3051-4f31-9297-c9f92fb03b46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:01:08.009290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee46a9-3051-4f31-9297-c9f92fb03b46 None admin] RBAC: Adding query filter params (role.id=ea0fd0c10f2f4e529270edbe3525d4c6, scope.project.id=6f4bcb81158c45cba0f1e51a2eb84361, user.id=7fa8a3b6ef4840a2b9ad4bdfddf7c615) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:01:08.009626 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee46a9-3051-4f31-9297-c9f92fb03b46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:01:08.011211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee46a9-3051-4f31-9297-c9f92fb03b46 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:01:08.017062 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 460/905] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 03:01:07 2022] GET /identity/v3/role_assignments?user.id=7fa8a3b6ef4840a2b9ad4bdfddf7c615&scope.project.id=6f4bcb81158c45cba0f1e51a2eb84361&role.id=ea0fd0c10f2f4e529270edbe3525d4c6 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:01:17.854839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=52 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:01:18.014341 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=21 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:03.668621 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76fe051-86f6-49c4-bde6-ed6305cc743f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:03.669242 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76fe051-86f6-49c4-bde6-ed6305cc743f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:03.669242 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76fe051-86f6-49c4-bde6-ed6305cc743f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:03.670635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 446/906] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:02:03.678605 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a508e5c-d486-4956-84be-e55355072bda None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:03.679127 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a508e5c-d486-4956-84be-e55355072bda None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:03.679306 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a508e5c-d486-4956-84be-e55355072bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:03.703832 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4a508e5c-d486-4956-84be-e55355072bda None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:02:03.705332 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4a508e5c-d486-4956-84be-e55355072bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:02:03.807229 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 461/907] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:02:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:02:04.151725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76fe051-86f6-49c4-bde6-ed6305cc743f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:04.305560 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76fe051-86f6-49c4-bde6-ed6305cc743f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:04.343438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8db81-3779-484b-a86d-ec669a579bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:04.344074 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8db81-3779-484b-a86d-ec669a579bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:04.344352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8db81-3779-484b-a86d-ec669a579bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:04.344621 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8db81-3779-484b-a86d-ec669a579bb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:04.356532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8db81-3779-484b-a86d-ec669a579bb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:04.358122 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8db81-3779-484b-a86d-ec669a579bb1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:04.376953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-d3d8db81-3779-484b-a86d-ec669a579bb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:04.376953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:04.376953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:04.376953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:04.376953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:04.376953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:04.376953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:04.376953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:04.376953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:04.376953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:04.376953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:04.376953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:04.376953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:02:04.376953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:02:04.376953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:02:04.376953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:02:04.378643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:04.378643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:04.378643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:04.378643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:04.378643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:04.378643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:04.378643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:04.378643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:04.378643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:04.378643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:04.378643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:04.378643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:04.378643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:04.378643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:04.378643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:04.378643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:04.378643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:02:04.379940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:02:04.379940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:04.379940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:02:04.379940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 447/908] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:04.389065 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a508e5c-d486-4956-84be-e55355072bda None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:04.530365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a508e5c-d486-4956-84be-e55355072bda None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:04.566379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:04.573590 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c92d1b-6a50-4ef8-bfa3-627dbce8430e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:04.574406 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c92d1b-6a50-4ef8-bfa3-627dbce8430e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:04.575036 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c92d1b-6a50-4ef8-bfa3-627dbce8430e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:04.575347 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c92d1b-6a50-4ef8-bfa3-627dbce8430e None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:04.575996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c92d1b-6a50-4ef8-bfa3-627dbce8430e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:04.576297 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c92d1b-6a50-4ef8-bfa3-627dbce8430e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:04.577826 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c92d1b-6a50-4ef8-bfa3-627dbce8430e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:04.596463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 462/909] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:04.602760 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8db81-3779-484b-a86d-ec669a579bb1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:04.673526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:04.732685 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8db81-3779-484b-a86d-ec669a579bb1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:04.779535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd33793-fecf-4f38-8744-a7aa62eb69ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:04.780331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd33793-fecf-4f38-8744-a7aa62eb69ef None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:04.780828 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd33793-fecf-4f38-8744-a7aa62eb69ef None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:04.781108 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd33793-fecf-4f38-8744-a7aa62eb69ef None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:04.782114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd33793-fecf-4f38-8744-a7aa62eb69ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:04.784478 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd33793-fecf-4f38-8744-a7aa62eb69ef None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:04.822225 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 448/910] 10.222.0.27 () {60 vars in 1132 bytes} [Mon Oct 17 03:02:04 2022] POST /identity/v3/users => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:02:07.458361 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16555fe-00d9-4fb0-a454-e090431260f7 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:07.458361 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16555fe-00d9-4fb0-a454-e090431260f7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:07.458361 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16555fe-00d9-4fb0-a454-e090431260f7 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:07.458361 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 463/911] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:02:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:02:07.464798 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1c068-d4ac-4d3a-a2c1-6a76e61ec23b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:07.465194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1c068-d4ac-4d3a-a2c1-6a76e61ec23b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:07.465376 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1c068-d4ac-4d3a-a2c1-6a76e61ec23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:07.485067 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-76a1c068-d4ac-4d3a-a2c1-6a76e61ec23b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:02:07.486368 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-76a1c068-d4ac-4d3a-a2c1-6a76e61ec23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:02:07.564903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 449/912] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:02:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:02:07.875039 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16555fe-00d9-4fb0-a454-e090431260f7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:07.998780 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16555fe-00d9-4fb0-a454-e090431260f7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:08.042371 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0672f-d5c0-41ce-bc40-5e856af3ea12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:08.043102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0672f-d5c0-41ce-bc40-5e856af3ea12 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:08.043386 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0672f-d5c0-41ce-bc40-5e856af3ea12 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:08.043659 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0672f-d5c0-41ce-bc40-5e856af3ea12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:08.055978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf0672f-d5c0-41ce-bc40-5e856af3ea12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:02:08.061164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-7cf0672f-d5c0-41ce-bc40-5e856af3ea12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:02:08.061164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:08.061164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:08.061164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:08.061164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:08.061164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:08.061164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:08.061164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:08.061164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:08.061164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:08.061164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:08.061164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:08.061164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:02:08.061164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:02:08.061164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:02:08.061164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:02:08.062163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:02:08.062163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:02:08.062163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:02:08.062163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:02:08.062163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:08.062163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:08.062163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:08.062163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:08.062163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:08.062163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:08.062163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:08.062163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:08.062163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:08.062163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:08.062163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:08.062163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:08.063200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:08.063200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:08.063200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:08.063200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:08.063200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:02:08.063200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:02:08.063200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:02:08.063200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:02:08.063200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:02:08.063200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:02:08.063200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:02:08.063200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:08.063200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 464/913] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Oct 17 03:02:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:08.069411 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1c068-d4ac-4d3a-a2c1-6a76e61ec23b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:08.115725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:08.189381 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1c068-d4ac-4d3a-a2c1-6a76e61ec23b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:08.228389 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fcb13-feb9-4232-83b0-bb22d40a6c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:08.229136 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fcb13-feb9-4232-83b0-bb22d40a6c62 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:08.229411 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fcb13-feb9-4232-83b0-bb22d40a6c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:08.229701 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fcb13-feb9-4232-83b0-bb22d40a6c62 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:08.230338 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860fcb13-feb9-4232-83b0-bb22d40a6c62 None admin] RBAC: Adding query filter params (name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:08.230711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860fcb13-feb9-4232-83b0-bb22d40a6c62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:08.232128 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860fcb13-feb9-4232-83b0-bb22d40a6c62 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:08.241033 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 450/914] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Oct 17 03:02:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:08.246636 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0672f-d5c0-41ce-bc40-5e856af3ea12 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:08.310661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:08.372293 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0672f-d5c0-41ce-bc40-5e856af3ea12 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:08.415009 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fbc04-434b-489a-8314-2ccd3e1078d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:08.416244 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fbc04-434b-489a-8314-2ccd3e1078d8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:08.416757 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fbc04-434b-489a-8314-2ccd3e1078d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:08.417239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fbc04-434b-489a-8314-2ccd3e1078d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:08.432234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fbc04-434b-489a-8314-2ccd3e1078d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:08.433699 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fbc04-434b-489a-8314-2ccd3e1078d8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:08.450978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-010fbc04-434b-489a-8314-2ccd3e1078d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:08.450978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:08.450978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:08.450978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:08.450978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:08.450978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:08.450978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:08.450978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:08.450978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:08.450978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:08.450978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:08.450978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:08.450978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:02:08.450978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:02:08.450978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:02:08.450978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:02:08.452075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:08.452075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:08.452075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:08.452075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:08.452075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:08.452075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:08.452075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:08.452075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:08.452075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:08.452075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:08.452075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:08.452075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:08.452075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:08.452075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:08.452075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:08.452075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:08.452075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:02:08.453661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:02:08.453661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:08.453661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:08.457704 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 465/915] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:02:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:02:08.459137 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fcb13-feb9-4232-83b0-bb22d40a6c62 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:08.579340 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:08.582631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fcb13-feb9-4232-83b0-bb22d40a6c62 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:08.619924 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d7c0d-ecdd-45db-b7b9-159c9016aca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:08.620539 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d7c0d-ecdd-45db-b7b9-159c9016aca9 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:08.620819 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d7c0d-ecdd-45db-b7b9-159c9016aca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:08.621090 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d7c0d-ecdd-45db-b7b9-159c9016aca9 None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:08.621736 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d7c0d-ecdd-45db-b7b9-159c9016aca9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:08.622041 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d7c0d-ecdd-45db-b7b9-159c9016aca9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:08.623459 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d7c0d-ecdd-45db-b7b9-159c9016aca9 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:08.641603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 451/916] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:08.642849 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fbc04-434b-489a-8314-2ccd3e1078d8 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:08.751268 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fbc04-434b-489a-8314-2ccd3e1078d8 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:08.753339 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:08.792527 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4319051-a299-47ae-b9c7-21095ad4e47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:08.795363 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4319051-a299-47ae-b9c7-21095ad4e47a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:08.795654 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4319051-a299-47ae-b9c7-21095ad4e47a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:08.795934 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4319051-a299-47ae-b9c7-21095ad4e47a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:08.805374 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4319051-a299-47ae-b9c7-21095ad4e47a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:08.806828 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4319051-a299-47ae-b9c7-21095ad4e47a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:08.819454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-f4319051-a299-47ae-b9c7-21095ad4e47a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 03:02:08.819454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:08.819454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:08.819454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:08.819454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:08.819454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:08.819454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:08.819454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:08.819454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:08.819454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:08.819454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:08.819454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:08.819454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:02:08.819454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:02:08.819454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:02:08.819454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:08.820695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:08.821712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:08.821712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:08.821712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:02:08.821712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:02:08.821712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:02:08.821712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:02:08.821712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:02:08.821712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:02:08.821712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 03:02:08.821712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:08.821712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 466/917] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:02:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:08.826844 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d7c0d-ecdd-45db-b7b9-159c9016aca9 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:08.954633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d7c0d-ecdd-45db-b7b9-159c9016aca9 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:08.993363 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad43c9e-b165-4619-bc0b-763842a84750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:08.994043 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad43c9e-b165-4619-bc0b-763842a84750 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:08.994327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad43c9e-b165-4619-bc0b-763842a84750 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:08.994638 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad43c9e-b165-4619-bc0b-763842a84750 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:08.995468 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad43c9e-b165-4619-bc0b-763842a84750 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:08.995778 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad43c9e-b165-4619-bc0b-763842a84750 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:08.997253 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad43c9e-b165-4619-bc0b-763842a84750 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:09.015526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 452/918] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Oct 17 03:02:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:09.022190 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4319051-a299-47ae-b9c7-21095ad4e47a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:09.127568 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4319051-a299-47ae-b9c7-21095ad4e47a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:09.162652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860911cd-a163-45a5-aa83-3a9cc57e138f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:09.163318 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860911cd-a163-45a5-aa83-3a9cc57e138f None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:09.163606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860911cd-a163-45a5-aa83-3a9cc57e138f None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:09.163917 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860911cd-a163-45a5-aa83-3a9cc57e138f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:09.176765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860911cd-a163-45a5-aa83-3a9cc57e138f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:09.178649 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860911cd-a163-45a5-aa83-3a9cc57e138f None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:09.197840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-860911cd-a163-45a5-aa83-3a9cc57e138f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:09.197840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:09.197840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:09.197840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:09.197840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:09.197840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:09.197840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:09.197840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:09.197840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:09.197840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:09.197840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:09.197840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:09.197840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:02:09.197840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:02:09.197840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:02:09.197840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:02:09.199116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:09.199116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:09.199116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:09.199116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:09.199116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:09.199116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:09.199116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:09.199116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:09.199116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:09.199116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:09.199116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:09.199116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:09.199116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:09.199116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:09.199116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:09.199116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:09.199116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:02:09.200028 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:02:09.200028 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:09.200028 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:09.200028 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 467/919] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:09.206425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad43c9e-b165-4619-bc0b-763842a84750 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:09.231824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:09.324307 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad43c9e-b165-4619-bc0b-763842a84750 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:09.363763 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb188da-703d-4140-bdb8-d93d6a07023d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:09.364381 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb188da-703d-4140-bdb8-d93d6a07023d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:09.364700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb188da-703d-4140-bdb8-d93d6a07023d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:09.365053 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb188da-703d-4140-bdb8-d93d6a07023d None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:09.365642 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb188da-703d-4140-bdb8-d93d6a07023d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:09.365943 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb188da-703d-4140-bdb8-d93d6a07023d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:09.367457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb188da-703d-4140-bdb8-d93d6a07023d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:09.383212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 453/920] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:09.387610 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860911cd-a163-45a5-aa83-3a9cc57e138f None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:09.405035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:09.500821 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860911cd-a163-45a5-aa83-3a9cc57e138f None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:09.542745 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6cb36-3b95-4ee9-8411-1a18b1d800e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:09.543428 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6cb36-3b95-4ee9-8411-1a18b1d800e8 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:09.543735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6cb36-3b95-4ee9-8411-1a18b1d800e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:09.544024 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6cb36-3b95-4ee9-8411-1a18b1d800e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:09.552753 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6cb36-3b95-4ee9-8411-1a18b1d800e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:09.554389 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6cb36-3b95-4ee9-8411-1a18b1d800e8 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:09.567457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-9cb6cb36-3b95-4ee9-8411-1a18b1d800e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:02:09.567457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:09.567457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:09.567457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:09.567457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:09.567457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:09.567457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:09.567457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:09.567457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:09.567457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:09.567457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:09.567457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:09.567457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:02:09.567457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:02:09.567457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:02:09.568676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:02:09.568676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:09.568676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:09.568676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:09.568676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:09.568676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:09.568676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:09.568676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:09.568676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:09.568676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:09.568676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:09.568676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:09.568676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:09.568676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:09.568676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:09.568676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:09.568676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:09.569557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:02:09.569557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:02:09.569557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:02:09.569557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:02:09.569557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:02:09.569557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:02:09.569557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:02:09.569557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:09.569557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 468/921] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 03:02:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:09.578588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb188da-703d-4140-bdb8-d93d6a07023d None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:09.676716 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:09.707471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb188da-703d-4140-bdb8-d93d6a07023d None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:09.750458 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcaa47f-8f5e-45c3-b444-49a909afa25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:09.751377 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcaa47f-8f5e-45c3-b444-49a909afa25d None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:09.751651 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcaa47f-8f5e-45c3-b444-49a909afa25d None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:09.751968 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcaa47f-8f5e-45c3-b444-49a909afa25d None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:09.752733 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcaa47f-8f5e-45c3-b444-49a909afa25d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:09.753130 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcaa47f-8f5e-45c3-b444-49a909afa25d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:09.754847 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcaa47f-8f5e-45c3-b444-49a909afa25d None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:09.770888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 454/922] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 03:02:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:09.777215 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6cb36-3b95-4ee9-8411-1a18b1d800e8 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:09.861393 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:09.893453 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6cb36-3b95-4ee9-8411-1a18b1d800e8 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:09.942471 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe7ca99-ef7f-4966-9422-40a73bea5edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:09.943242 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7ca99-ef7f-4966-9422-40a73bea5edc None admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:09.943525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7ca99-ef7f-4966-9422-40a73bea5edc None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:09.943794 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe7ca99-ef7f-4966-9422-40a73bea5edc None admin] PATH_INFO: `/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/4cc2655cc32e4d63a7000ef41bcbcfea/roles/78dc08989b32455389b21096f5d57e17` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:09.961651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7ca99-ef7f-4966-9422-40a73bea5edc None admin] RBAC: Authorizing `identity:create_grant(project_id=89b0ee95955f465eb9ee5b8ae337d9c2, user_id=4cc2655cc32e4d63a7000ef41bcbcfea, role_id=78dc08989b32455389b21096f5d57e17)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:09.963795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe7ca99-ef7f-4966-9422-40a73bea5edc None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:09.976877 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 469/923] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 03:02:09 2022] PUT /identity/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/4cc2655cc32e4d63a7000ef41bcbcfea/roles/78dc08989b32455389b21096f5d57e17 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:02:12.168779 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c96b3-950a-484d-848d-fa3412f41f66 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:12.169393 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c96b3-950a-484d-848d-fa3412f41f66 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:12.169393 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c96b3-950a-484d-848d-fa3412f41f66 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:12.170654 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 455/924] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:02:12.177153 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a1889-8115-4363-9cae-202d18813828 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:12.177654 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a1889-8115-4363-9cae-202d18813828 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:12.177744 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a1889-8115-4363-9cae-202d18813828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:12.197372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-051a1889-8115-4363-9cae-202d18813828 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:02:12.198588 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-051a1889-8115-4363-9cae-202d18813828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:02:12.274331 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 470/925] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:02:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:02:12.550490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c96b3-950a-484d-848d-fa3412f41f66 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:12.668292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c96b3-950a-484d-848d-fa3412f41f66 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:12.711732 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0cc3d-c4ea-4d3a-b03e-dbfcbdde43b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:12.712400 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0cc3d-c4ea-4d3a-b03e-dbfcbdde43b6 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:12.712680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0cc3d-c4ea-4d3a-b03e-dbfcbdde43b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:12.712951 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0cc3d-c4ea-4d3a-b03e-dbfcbdde43b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:12.720184 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:12.724852 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0cc3d-c4ea-4d3a-b03e-dbfcbdde43b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:02:12.730234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-ecb0cc3d-c4ea-4d3a-b03e-dbfcbdde43b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:02:12.730234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:12.730234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:12.730234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:12.730234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:12.730234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:12.730234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:12.730234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:12.730234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:12.730234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:12.730234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:12.730234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:12.730234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:02:12.730234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:02:12.730234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:02:12.730234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:02:12.731503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:02:12.731503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:02:12.731503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:02:12.731503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:02:12.731503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:12.731503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:12.731503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:12.731503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:12.731503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:12.731503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:12.731503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:12.731503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:12.731503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:12.731503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:12.731503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:12.731503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:12.732637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:12.732637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:12.732637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:12.732637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:12.732637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:02:12.732637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:02:12.732637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:02:12.732637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:02:12.732637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:02:12.732637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:02:12.732637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:02:12.732637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:02:12.732637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 456/926] 10.222.0.27 () {56 vars in 1102 bytes} [Mon Oct 17 03:02:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:12.738246 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a1889-8115-4363-9cae-202d18813828 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:12.798043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:12.846268 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a1889-8115-4363-9cae-202d18813828 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:12.885146 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa1895-4abc-4808-ae31-d6d385f8c993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:12.885917 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa1895-4abc-4808-ae31-d6d385f8c993 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:12.886309 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa1895-4abc-4808-ae31-d6d385f8c993 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:12.886627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa1895-4abc-4808-ae31-d6d385f8c993 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:12.887444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa1895-4abc-4808-ae31-d6d385f8c993 None admin] RBAC: Adding query filter params (name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:12.887839 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa1895-4abc-4808-ae31-d6d385f8c993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:12.889441 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa1895-4abc-4808-ae31-d6d385f8c993 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:12.900270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 471/927] 10.222.0.27 () {56 vars in 1103 bytes} [Mon Oct 17 03:02:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:02:12.906410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0cc3d-c4ea-4d3a-b03e-dbfcbdde43b6 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:13.028313 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0cc3d-c4ea-4d3a-b03e-dbfcbdde43b6 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:13.068008 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fae247-3f72-4ccf-bde9-35c4a95f987c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:13.068708 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fae247-3f72-4ccf-bde9-35c4a95f987c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:13.069000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fae247-3f72-4ccf-bde9-35c4a95f987c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:13.069343 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fae247-3f72-4ccf-bde9-35c4a95f987c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:13.080029 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fae247-3f72-4ccf-bde9-35c4a95f987c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:13.082172 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fae247-3f72-4ccf-bde9-35c4a95f987c None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:13.097909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-06fae247-3f72-4ccf-bde9-35c4a95f987c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:13.097909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:13.097909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:13.097909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:13.097909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:13.097909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:13.097909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:13.097909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:13.097909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:13.097909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:13.097909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:13.097909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:13.097909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:02:13.097909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:02:13.097909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:02:13.097909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:02:13.099212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:13.099212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:13.099212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:13.099212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:13.099212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:13.099212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:13.099212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:13.099212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:13.099212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:13.099212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:13.099212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:13.099212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:13.099212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:13.099212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:13.099212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:13.099212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:13.099212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:02:13.100502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:02:13.100502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:13.100502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:02:13.104245 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 457/928] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:13.105550 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa1895-4abc-4808-ae31-d6d385f8c993 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:13.229475 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa1895-4abc-4808-ae31-d6d385f8c993 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:13.251750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:13.273231 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4034ce-875e-4687-90f0-28989f8356c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:13.273840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4034ce-875e-4687-90f0-28989f8356c3 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:13.274633 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4034ce-875e-4687-90f0-28989f8356c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:13.275075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4034ce-875e-4687-90f0-28989f8356c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:13.275750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4034ce-875e-4687-90f0-28989f8356c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:13.276064 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4034ce-875e-4687-90f0-28989f8356c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:13.277659 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4034ce-875e-4687-90f0-28989f8356c3 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:13.293911 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 472/929] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:13.300760 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fae247-3f72-4ccf-bde9-35c4a95f987c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:13.359471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:13.428894 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fae247-3f72-4ccf-bde9-35c4a95f987c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:13.470752 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf4601-6f19-4c55-97a1-6be5fa2839e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:13.471875 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf4601-6f19-4c55-97a1-6be5fa2839e1 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:13.472373 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf4601-6f19-4c55-97a1-6be5fa2839e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:13.472731 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf4601-6f19-4c55-97a1-6be5fa2839e1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:13.483802 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf4601-6f19-4c55-97a1-6be5fa2839e1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:13.485759 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf4601-6f19-4c55-97a1-6be5fa2839e1 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:13.500780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-04cf4601-6f19-4c55-97a1-6be5fa2839e1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 03:02:13.500780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:13.500780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:13.500780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:13.500780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:13.500780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:13.500780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:13.500780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:13.500780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:13.500780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:13.500780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:13.500780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:13.500780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:02:13.500780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:02:13.500780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:02:13.500780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:13.502038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:13.504274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:13.504274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:13.504274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:02:13.504274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:02:13.504274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:02:13.504274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:02:13.504274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:02:13.504274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:02:13.504274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 03:02:13.504274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:02:13.504274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 458/930] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:02:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:02:13.509328 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4034ce-875e-4687-90f0-28989f8356c3 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:13.630710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4034ce-875e-4687-90f0-28989f8356c3 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:13.669078 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129c089-8f28-47e4-b30b-33c50ad0bf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:13.669752 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129c089-8f28-47e4-b30b-33c50ad0bf40 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:13.670068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129c089-8f28-47e4-b30b-33c50ad0bf40 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:13.670413 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129c089-8f28-47e4-b30b-33c50ad0bf40 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:13.671264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129c089-8f28-47e4-b30b-33c50ad0bf40 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:13.671598 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129c089-8f28-47e4-b30b-33c50ad0bf40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:13.673215 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129c089-8f28-47e4-b30b-33c50ad0bf40 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:13.693626 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 473/931] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Oct 17 03:02:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:13.699965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf4601-6f19-4c55-97a1-6be5fa2839e1 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:13.829021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf4601-6f19-4c55-97a1-6be5fa2839e1 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:13.841685 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:13.870791 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e538f9-15e9-42f4-b6ab-654ec2e19895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:13.871432 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e538f9-15e9-42f4-b6ab-654ec2e19895 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:13.871712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e538f9-15e9-42f4-b6ab-654ec2e19895 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:13.871983 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e538f9-15e9-42f4-b6ab-654ec2e19895 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:13.948962 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e538f9-15e9-42f4-b6ab-654ec2e19895 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:13.950380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e538f9-15e9-42f4-b6ab-654ec2e19895 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:14.063896 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 459/932] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 03:02:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 03:02:14.070354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129c089-8f28-47e4-b30b-33c50ad0bf40 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:14.110238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:14.194272 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129c089-8f28-47e4-b30b-33c50ad0bf40 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:14.230779 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8038c5-bc86-49c2-b5db-503fcad1d42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:14.231531 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8038c5-bc86-49c2-b5db-503fcad1d42a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:14.231900 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8038c5-bc86-49c2-b5db-503fcad1d42a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:14.232223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8038c5-bc86-49c2-b5db-503fcad1d42a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:14.242566 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8038c5-bc86-49c2-b5db-503fcad1d42a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:14.244040 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8038c5-bc86-49c2-b5db-503fcad1d42a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:14.260634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-0b8038c5-bc86-49c2-b5db-503fcad1d42a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:14.260634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:14.260634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:14.260634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:14.260634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:14.260634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:14.260634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:14.260634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:14.260634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:14.260634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:14.260634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:14.260634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:14.260634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:02:14.260634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:02:14.260634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:02:14.260634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:02:14.262364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:14.262364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:14.262364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:14.262364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:14.262364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:14.262364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:14.262364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:14.262364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:14.262364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:14.262364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:14.262364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:14.262364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:14.262364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:14.262364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:14.262364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:14.262364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:14.262364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:02:14.263622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:02:14.263622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:14.263622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:14.263622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 474/933] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:02:14.271464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e538f9-15e9-42f4-b6ab-654ec2e19895 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:14.284862 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:14.383837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e538f9-15e9-42f4-b6ab-654ec2e19895 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:14.420336 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbcf03-e39e-46f5-ba70-6243e082820f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:14.420947 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbcf03-e39e-46f5-ba70-6243e082820f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:14.421224 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbcf03-e39e-46f5-ba70-6243e082820f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:14.421494 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbcf03-e39e-46f5-ba70-6243e082820f None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:14.422148 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbcf03-e39e-46f5-ba70-6243e082820f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:14.422449 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbcf03-e39e-46f5-ba70-6243e082820f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:14.423842 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbcf03-e39e-46f5-ba70-6243e082820f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:14.437188 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 460/934] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:14.443760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8038c5-bc86-49c2-b5db-503fcad1d42a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:14.548098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:14.566553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8038c5-bc86-49c2-b5db-503fcad1d42a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:14.603951 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d775df4-52f3-4de1-890a-c079a4fd6f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:14.604582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d775df4-52f3-4de1-890a-c079a4fd6f9a None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:14.604859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d775df4-52f3-4de1-890a-c079a4fd6f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:14.605145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d775df4-52f3-4de1-890a-c079a4fd6f9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:14.613147 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d775df4-52f3-4de1-890a-c079a4fd6f9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:14.614591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d775df4-52f3-4de1-890a-c079a4fd6f9a None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:14.625606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-7d775df4-52f3-4de1-890a-c079a4fd6f9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:02:14.625606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:14.625606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:14.625606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:14.625606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:14.625606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:14.625606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:14.625606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:14.625606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:14.625606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:14.625606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:14.625606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:14.625606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:02:14.625606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:02:14.625606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:02:14.626702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:02:14.626702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:14.626702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:14.626702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:14.626702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:14.626702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:14.626702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:14.626702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:14.626702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:14.626702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:14.626702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:14.626702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:14.626702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:14.626702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:14.626702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:14.626702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:14.626702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:14.627667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:02:14.627667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:02:14.627667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:02:14.627667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:02:14.627667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:02:14.627667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:02:14.627667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:02:14.627667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:14.627667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 475/935] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 03:02:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:14.633132 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbcf03-e39e-46f5-ba70-6243e082820f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:14.715622 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:14.755206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbcf03-e39e-46f5-ba70-6243e082820f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:14.794552 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a53d3e-3aa5-4fe2-bbdb-7f7acdab2d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:14.795255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a53d3e-3aa5-4fe2-bbdb-7f7acdab2d00 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:14.795651 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a53d3e-3aa5-4fe2-bbdb-7f7acdab2d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:14.795840 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a53d3e-3aa5-4fe2-bbdb-7f7acdab2d00 None admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:14.796613 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a53d3e-3aa5-4fe2-bbdb-7f7acdab2d00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:14.796920 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a53d3e-3aa5-4fe2-bbdb-7f7acdab2d00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:14.798404 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a53d3e-3aa5-4fe2-bbdb-7f7acdab2d00 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:14.811991 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 461/936] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 03:02:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:02:14.819122 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d775df4-52f3-4de1-890a-c079a4fd6f9a None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:14.940329 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d775df4-52f3-4de1-890a-c079a4fd6f9a None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:14.979601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c36bd-b4b5-4a2b-9f6b-23326026fb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:14.980383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c36bd-b4b5-4a2b-9f6b-23326026fb62 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:14.980782 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c36bd-b4b5-4a2b-9f6b-23326026fb62 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:14.981079 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c36bd-b4b5-4a2b-9f6b-23326026fb62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:14.981949 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c36bd-b4b5-4a2b-9f6b-23326026fb62 None admin] RBAC: Adding query filter params (role.id=78dc08989b32455389b21096f5d57e17, scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2, user.id=4cc2655cc32e4d63a7000ef41bcbcfea) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:14.982263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c36bd-b4b5-4a2b-9f6b-23326026fb62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:14.983885 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c36bd-b4b5-4a2b-9f6b-23326026fb62 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:14.991863 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 476/937] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 03:02:14 2022] GET /identity/v3/role_assignments?user.id=4cc2655cc32e4d63a7000ef41bcbcfea&scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2&role.id=78dc08989b32455389b21096f5d57e17 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:17.202136 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52605ce0-c440-4b61-8025-cfabb2360636 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:17.202757 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52605ce0-c440-4b61-8025-cfabb2360636 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:17.202757 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52605ce0-c440-4b61-8025-cfabb2360636 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:17.204022 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 462/938] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:02:17.210548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda33071-3d0a-4d34-a1bb-baf3c86b80e4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:17.210969 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda33071-3d0a-4d34-a1bb-baf3c86b80e4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:17.211126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda33071-3d0a-4d34-a1bb-baf3c86b80e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:17.230263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-cda33071-3d0a-4d34-a1bb-baf3c86b80e4 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:02:17.231587 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cda33071-3d0a-4d34-a1bb-baf3c86b80e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:02:17.234722 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:17.307760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 477/939] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:02:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:02:17.590719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52605ce0-c440-4b61-8025-cfabb2360636 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:17.700073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52605ce0-c440-4b61-8025-cfabb2360636 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:17.735494 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:17.742223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc290f5a-830f-4e7a-9538-c30570fe053c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:17.743124 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc290f5a-830f-4e7a-9538-c30570fe053c None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:17.743618 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc290f5a-830f-4e7a-9538-c30570fe053c None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:17.743892 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc290f5a-830f-4e7a-9538-c30570fe053c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:17.755482 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc290f5a-830f-4e7a-9538-c30570fe053c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:02:17.760903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-fc290f5a-830f-4e7a-9538-c30570fe053c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:02:17.760903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:17.760903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:17.760903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:17.760903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:17.760903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:17.760903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:17.760903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:17.760903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:17.760903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:17.760903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:17.760903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:17.760903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:02:17.760903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:02:17.760903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:02:17.760903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:02:17.762060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:02:17.762060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:02:17.762060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:02:17.762060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:02:17.762060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:17.762060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:17.762060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:17.762060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:17.762060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:17.762060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:17.762060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:17.762060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:17.762060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:17.762060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:17.762060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:17.762060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:17.763292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:17.763292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:17.763292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:17.763292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:17.763292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:02:17.763292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:02:17.763292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:02:17.763292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:02:17.763292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:02:17.763292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:02:17.763292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:02:17.763292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:02:17.763292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 463/940] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 03:02:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:17.768469 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda33071-3d0a-4d34-a1bb-baf3c86b80e4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:17.872481 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda33071-3d0a-4d34-a1bb-baf3c86b80e4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:17.913767 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec63b6-7f6d-4e5d-9c2a-4cc8b96c7f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:17.914848 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec63b6-7f6d-4e5d-9c2a-4cc8b96c7f00 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:17.915225 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec63b6-7f6d-4e5d-9c2a-4cc8b96c7f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:17.915573 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecec63b6-7f6d-4e5d-9c2a-4cc8b96c7f00 None admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:17.916881 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec63b6-7f6d-4e5d-9c2a-4cc8b96c7f00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:17.917206 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec63b6-7f6d-4e5d-9c2a-4cc8b96c7f00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:17.919565 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecec63b6-7f6d-4e5d-9c2a-4cc8b96c7f00 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:17.930176 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 478/941] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 03:02:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:17.937890 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc290f5a-830f-4e7a-9538-c30570fe053c None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:18.058024 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc290f5a-830f-4e7a-9538-c30570fe053c None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:18.093438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25a52e-1e83-4cd2-bf5c-94cbc7fcc498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:18.094058 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25a52e-1e83-4cd2-bf5c-94cbc7fcc498 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:18.094343 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25a52e-1e83-4cd2-bf5c-94cbc7fcc498 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:18.094623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e25a52e-1e83-4cd2-bf5c-94cbc7fcc498 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:18.106694 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25a52e-1e83-4cd2-bf5c-94cbc7fcc498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:18.108219 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e25a52e-1e83-4cd2-bf5c-94cbc7fcc498 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:18.123206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-1e25a52e-1e83-4cd2-bf5c-94cbc7fcc498 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:18.123206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:18.123206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:18.123206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:18.123206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:18.123206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:18.123206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:18.123206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:18.123206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:18.123206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:18.123206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:18.123206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:18.123206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:02:18.123206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:02:18.123206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:02:18.123206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:02:18.124573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:18.124573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:18.124573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:18.124573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:18.124573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:18.124573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:18.124573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:18.124573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:18.124573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:18.124573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:18.124573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:18.124573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:18.124573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:18.124573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:18.124573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:18.124573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:18.124573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:02:18.125763 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:02:18.125763 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:18.125763 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:02:18.125763 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 464/942] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:18.130053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec63b6-7f6d-4e5d-9c2a-4cc8b96c7f00 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:18.133026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:18.255701 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecec63b6-7f6d-4e5d-9c2a-4cc8b96c7f00 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:18.302760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e721bf6-0f7b-4964-845d-857bf9149761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:18.303653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e721bf6-0f7b-4964-845d-857bf9149761 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:18.304007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e721bf6-0f7b-4964-845d-857bf9149761 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:18.304470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e721bf6-0f7b-4964-845d-857bf9149761 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:18.305580 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e721bf6-0f7b-4964-845d-857bf9149761 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:18.306127 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e721bf6-0f7b-4964-845d-857bf9149761 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:18.307882 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e721bf6-0f7b-4964-845d-857bf9149761 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:18.324495 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 479/943] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:18.332057 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25a52e-1e83-4cd2-bf5c-94cbc7fcc498 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:18.385858 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:18.450156 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e25a52e-1e83-4cd2-bf5c-94cbc7fcc498 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:18.485914 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cd925-bc8e-4794-b9b9-7654bdc406c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:18.486541 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cd925-bc8e-4794-b9b9-7654bdc406c8 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:18.486859 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cd925-bc8e-4794-b9b9-7654bdc406c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:18.487215 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cd925-bc8e-4794-b9b9-7654bdc406c8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:18.495178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8cd925-bc8e-4794-b9b9-7654bdc406c8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:18.496611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8cd925-bc8e-4794-b9b9-7654bdc406c8 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:18.508406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-dd8cd925-bc8e-4794-b9b9-7654bdc406c8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 03:02:18.508406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:18.508406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:18.508406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:18.508406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:18.508406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:18.508406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:18.508406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:18.508406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:18.508406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:18.508406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:18.508406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:18.508406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:02:18.508406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:02:18.508406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:02:18.508406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:18.509394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:18.510589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:18.510589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:18.510589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:02:18.510589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:02:18.510589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:02:18.510589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:02:18.510589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:02:18.510589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:02:18.510589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 03:02:18.510589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:02:18.510589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 465/944] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:02:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:02:18.516231 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e721bf6-0f7b-4964-845d-857bf9149761 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:18.585173 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:18.629143 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e721bf6-0f7b-4964-845d-857bf9149761 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:18.672133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab533fdb-964c-413f-9b36-8ee403ca0204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:18.672937 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab533fdb-964c-413f-9b36-8ee403ca0204 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:18.673428 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab533fdb-964c-413f-9b36-8ee403ca0204 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:18.673740 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab533fdb-964c-413f-9b36-8ee403ca0204 None admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:18.674556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab533fdb-964c-413f-9b36-8ee403ca0204 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:18.675050 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab533fdb-964c-413f-9b36-8ee403ca0204 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:18.677182 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab533fdb-964c-413f-9b36-8ee403ca0204 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:18.698916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 480/945] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Oct 17 03:02:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:18.704789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cd925-bc8e-4794-b9b9-7654bdc406c8 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:18.815986 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cd925-bc8e-4794-b9b9-7654bdc406c8 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:18.817831 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:18.856266 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b09e7e-40e1-4ba4-af30-6b4e4e4419a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:18.856908 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b09e7e-40e1-4ba4-af30-6b4e4e4419a2 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:18.857195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b09e7e-40e1-4ba4-af30-6b4e4e4419a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:18.857471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b09e7e-40e1-4ba4-af30-6b4e4e4419a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:18.869770 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b09e7e-40e1-4ba4-af30-6b4e4e4419a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:18.871240 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b09e7e-40e1-4ba4-af30-6b4e4e4419a2 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:18.887268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-84b09e7e-40e1-4ba4-af30-6b4e4e4419a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:18.887268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:18.887268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:18.887268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:18.887268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:18.887268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:18.887268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:18.887268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:18.887268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:18.887268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:18.887268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:18.887268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:18.887268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:02:18.887268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:02:18.887268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:02:18.887268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:02:18.888702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:18.888702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:18.888702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:18.888702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:18.888702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:18.888702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:18.888702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:18.888702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:18.888702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:18.888702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:18.888702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:18.888702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:18.888702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:18.888702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:18.888702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:18.888702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:18.888702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:02:18.889903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:02:18.889903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:18.889903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:02:18.889903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 466/946] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:02:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:18.894750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab533fdb-964c-413f-9b36-8ee403ca0204 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:19.016051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab533fdb-964c-413f-9b36-8ee403ca0204 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:19.037263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:19.055169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfe5a2-c1a7-4d9c-82ef-191f74e02d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:19.055778 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfe5a2-c1a7-4d9c-82ef-191f74e02d6c None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:19.056058 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfe5a2-c1a7-4d9c-82ef-191f74e02d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:19.056324 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfe5a2-c1a7-4d9c-82ef-191f74e02d6c None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:19.056958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfe5a2-c1a7-4d9c-82ef-191f74e02d6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:19.057254 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfe5a2-c1a7-4d9c-82ef-191f74e02d6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:19.058623 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfe5a2-c1a7-4d9c-82ef-191f74e02d6c None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:19.070357 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 481/947] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:19.076081 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b09e7e-40e1-4ba4-af30-6b4e4e4419a2 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:19.180698 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b09e7e-40e1-4ba4-af30-6b4e4e4419a2 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:19.224255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30271bdb-d844-4c5d-8cb8-2b1c05ae5f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:19.225021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30271bdb-d844-4c5d-8cb8-2b1c05ae5f8f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:19.225341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30271bdb-d844-4c5d-8cb8-2b1c05ae5f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:19.225667 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30271bdb-d844-4c5d-8cb8-2b1c05ae5f8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:19.234466 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30271bdb-d844-4c5d-8cb8-2b1c05ae5f8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:19.236470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30271bdb-d844-4c5d-8cb8-2b1c05ae5f8f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:19.249511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-30271bdb-d844-4c5d-8cb8-2b1c05ae5f8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:02:19.249511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:19.249511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:19.249511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:19.249511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:19.249511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:19.249511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:19.249511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:19.249511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:19.249511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:19.249511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:19.249511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:19.249511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:02:19.249511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:02:19.249511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:02:19.256021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:02:19.256021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:19.256021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:19.256021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:19.256021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:19.256021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:19.256021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:19.256021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:19.256021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:19.256021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:19.256021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:19.256021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:19.256021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:19.256021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:19.256021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:19.256021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:19.256021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:19.257157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:02:19.257157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:02:19.257157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:02:19.257157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:02:19.257157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:02:19.257157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:02:19.257157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:02:19.257157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:02:19.257157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 467/948] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 03:02:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:19.257736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfe5a2-c1a7-4d9c-82ef-191f74e02d6c None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:19.374089 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfe5a2-c1a7-4d9c-82ef-191f74e02d6c None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:19.414647 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1526e0-873c-4102-9f68-bd49d948512e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:19.415320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1526e0-873c-4102-9f68-bd49d948512e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:19.415618 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1526e0-873c-4102-9f68-bd49d948512e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:19.415945 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1526e0-873c-4102-9f68-bd49d948512e None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:19.416705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1526e0-873c-4102-9f68-bd49d948512e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:19.417079 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1526e0-873c-4102-9f68-bd49d948512e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:19.418530 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1526e0-873c-4102-9f68-bd49d948512e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:19.431242 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 482/949] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 03:02:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:19.437566 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30271bdb-d844-4c5d-8cb8-2b1c05ae5f8f None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:19.458607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:19.556264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30271bdb-d844-4c5d-8cb8-2b1c05ae5f8f None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:19.590926 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75676e1-2e3a-467a-8e83-d181e72b8644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:19.591581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75676e1-2e3a-467a-8e83-d181e72b8644 None admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:19.591858 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75676e1-2e3a-467a-8e83-d181e72b8644 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:19.592129 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75676e1-2e3a-467a-8e83-d181e72b8644 None admin] PATH_INFO: `/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/4cc2655cc32e4d63a7000ef41bcbcfea/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:19.594030 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75676e1-2e3a-467a-8e83-d181e72b8644 None admin] RBAC: Authorizing `identity:create_grant(project_id=89b0ee95955f465eb9ee5b8ae337d9c2, user_id=4cc2655cc32e4d63a7000ef41bcbcfea, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:19.595566 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75676e1-2e3a-467a-8e83-d181e72b8644 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:19.609895 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 468/950] 10.222.0.27 () {58 vars in 1439 bytes} [Mon Oct 17 03:02:19 2022] PUT /identity/v3/projects/89b0ee95955f465eb9ee5b8ae337d9c2/users/4cc2655cc32e4d63a7000ef41bcbcfea/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:02:22.219321 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e69a4-4827-4667-b1e7-d3ac95c2aa24 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:22.219919 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e69a4-4827-4667-b1e7-d3ac95c2aa24 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:22.220018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e69a4-4827-4667-b1e7-d3ac95c2aa24 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:22.221516 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 483/951] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:02:22.228581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f8cba-ccda-4411-b691-845bac49fcba None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:22.229026 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f8cba-ccda-4411-b691-845bac49fcba None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:22.229116 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f8cba-ccda-4411-b691-845bac49fcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:22.247508 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-395f8cba-ccda-4411-b691-845bac49fcba None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:02:22.248683 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-395f8cba-ccda-4411-b691-845bac49fcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:02:22.322422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 469/952] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:02:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:02:22.618655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e69a4-4827-4667-b1e7-d3ac95c2aa24 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:22.667317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:22.727141 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e69a4-4827-4667-b1e7-d3ac95c2aa24 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:22.764873 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c61b30-1727-4462-94ec-ded58dabdc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:22.765515 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c61b30-1727-4462-94ec-ded58dabdc24 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:22.765803 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c61b30-1727-4462-94ec-ded58dabdc24 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:22.766142 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c61b30-1727-4462-94ec-ded58dabdc24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:22.778789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c61b30-1727-4462-94ec-ded58dabdc24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:02:22.785090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-02c61b30-1727-4462-94ec-ded58dabdc24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:02:22.785090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:22.785090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:22.785090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:22.785090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:22.785090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:22.785090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:22.785090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:22.785090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:22.785090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:22.785090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:22.785090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:22.785090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:02:22.785090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:02:22.785090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:02:22.785090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:02:22.786171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:02:22.786171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:02:22.786171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:02:22.786171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:02:22.786171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:22.786171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:22.786171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:22.786171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:22.786171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:22.786171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:22.786171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:22.786171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:22.786171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:22.786171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:22.786171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:22.786171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:22.787258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:22.787258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:22.787258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:22.787258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:22.787258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:02:22.787258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:02:22.787258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:02:22.787258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:02:22.787258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:02:22.787258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:02:22.787258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:02:22.787258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:22.787258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 484/953] 10.222.0.27 () {56 vars in 1096 bytes} [Mon Oct 17 03:02:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:22.792885 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f8cba-ccda-4411-b691-845bac49fcba None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:22.796952 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:22.912650 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f8cba-ccda-4411-b691-845bac49fcba None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:22.955388 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd329b0f-e168-4db0-9d4e-4c8bbbe3fc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:22.956005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd329b0f-e168-4db0-9d4e-4c8bbbe3fc86 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:22.957208 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd329b0f-e168-4db0-9d4e-4c8bbbe3fc86 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:22.957612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd329b0f-e168-4db0-9d4e-4c8bbbe3fc86 None admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:22.958922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd329b0f-e168-4db0-9d4e-4c8bbbe3fc86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:22.959311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd329b0f-e168-4db0-9d4e-4c8bbbe3fc86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:22.961853 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd329b0f-e168-4db0-9d4e-4c8bbbe3fc86 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:22.973188 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 470/954] 10.222.0.27 () {56 vars in 1099 bytes} [Mon Oct 17 03:02:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:22.979053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c61b30-1727-4462-94ec-ded58dabdc24 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:23.082701 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:23.085513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c61b30-1727-4462-94ec-ded58dabdc24 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:23.120947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b1893-172f-4c1a-93ec-1993b2606121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:23.121624 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b1893-172f-4c1a-93ec-1993b2606121 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:23.121871 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b1893-172f-4c1a-93ec-1993b2606121 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:23.122137 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b1893-172f-4c1a-93ec-1993b2606121 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:23.133028 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b1893-172f-4c1a-93ec-1993b2606121 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:23.134463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b1893-172f-4c1a-93ec-1993b2606121 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:23.151524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-d51b1893-172f-4c1a-93ec-1993b2606121 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:23.151524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:23.151524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:23.151524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:23.151524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:23.151524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:23.151524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:23.151524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:23.151524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:23.151524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:23.151524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:23.151524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:23.151524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:02:23.151524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:02:23.151524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:02:23.151524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:02:23.152992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:23.152992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:23.152992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:23.152992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:23.152992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:23.152992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:23.152992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:23.152992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:23.152992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:23.152992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:23.152992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:23.152992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:23.152992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:23.152992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:23.152992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:23.152992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:23.152992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:02:23.154147 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:02:23.154147 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:23.154147 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:23.154147 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 485/955] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:23.159827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd329b0f-e168-4db0-9d4e-4c8bbbe3fc86 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:23.238375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:23.277019 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd329b0f-e168-4db0-9d4e-4c8bbbe3fc86 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:23.312598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d6f77-b011-4470-8c7d-9d332dd0061b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:23.313232 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d6f77-b011-4470-8c7d-9d332dd0061b None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:23.313509 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d6f77-b011-4470-8c7d-9d332dd0061b None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:23.313785 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d6f77-b011-4470-8c7d-9d332dd0061b None admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:23.314489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d6f77-b011-4470-8c7d-9d332dd0061b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:23.314791 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d6f77-b011-4470-8c7d-9d332dd0061b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:23.316413 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d6f77-b011-4470-8c7d-9d332dd0061b None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:23.329558 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 471/956] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:23.336905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b1893-172f-4c1a-93ec-1993b2606121 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:23.440523 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b1893-172f-4c1a-93ec-1993b2606121 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:23.475951 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f370a-2297-40ca-9a0f-1424d622c954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:23.476605 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f370a-2297-40ca-9a0f-1424d622c954 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:23.476883 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f370a-2297-40ca-9a0f-1424d622c954 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:23.477154 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f370a-2297-40ca-9a0f-1424d622c954 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:23.485344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f370a-2297-40ca-9a0f-1424d622c954 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:23.486792 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f370a-2297-40ca-9a0f-1424d622c954 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:23.498990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-b19f370a-2297-40ca-9a0f-1424d622c954 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 03:02:23.498990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:23.498990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:23.498990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:23.498990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:23.498990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:23.498990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:23.498990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:23.498990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:23.498990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:23.498990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:23.498990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:23.498990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:02:23.498990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:02:23.498990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:02:23.498990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:23.500174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:23.501397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:23.501397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:23.501397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:02:23.501397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:02:23.501397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:02:23.501397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:02:23.501397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:02:23.501397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:02:23.501397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 03:02:23.501397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:23.502243 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 486/957] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:02:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:23.506149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d6f77-b011-4470-8c7d-9d332dd0061b None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:23.613126 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d6f77-b011-4470-8c7d-9d332dd0061b None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:23.641010 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:23.650963 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466b5a5-9390-42e5-a78f-32f8c47f3f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:23.651581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466b5a5-9390-42e5-a78f-32f8c47f3f44 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:23.651865 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466b5a5-9390-42e5-a78f-32f8c47f3f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:23.652140 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466b5a5-9390-42e5-a78f-32f8c47f3f44 None admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:23.652873 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466b5a5-9390-42e5-a78f-32f8c47f3f44 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:23.653181 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466b5a5-9390-42e5-a78f-32f8c47f3f44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:23.655265 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466b5a5-9390-42e5-a78f-32f8c47f3f44 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:23.673435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 472/958] 10.222.0.27 () {56 vars in 1143 bytes} [Mon Oct 17 03:02:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:02:23.679821 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f370a-2297-40ca-9a0f-1424d622c954 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:23.695631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:23.782913 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f370a-2297-40ca-9a0f-1424d622c954 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:23.819812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51787616-d3c4-459d-ad55-50b4a7f4e02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:23.820675 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51787616-d3c4-459d-ad55-50b4a7f4e02e None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:23.820950 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51787616-d3c4-459d-ad55-50b4a7f4e02e None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:23.821223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51787616-d3c4-459d-ad55-50b4a7f4e02e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:23.889846 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51787616-d3c4-459d-ad55-50b4a7f4e02e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:23.891506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51787616-d3c4-459d-ad55-50b4a7f4e02e None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:23.906587 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:23.993298 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 487/959] 10.222.0.27 () {58 vars in 1282 bytes} [Mon Oct 17 03:02:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 03:02:23.999149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466b5a5-9390-42e5-a78f-32f8c47f3f44 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:24.094054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466b5a5-9390-42e5-a78f-32f8c47f3f44 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:24.125974 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab65929-64d9-4a69-878a-41ac5d460f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:24.126581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab65929-64d9-4a69-878a-41ac5d460f81 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:24.126853 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab65929-64d9-4a69-878a-41ac5d460f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:24.127150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab65929-64d9-4a69-878a-41ac5d460f81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:24.136855 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab65929-64d9-4a69-878a-41ac5d460f81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:24.138275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab65929-64d9-4a69-878a-41ac5d460f81 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:24.152544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-9ab65929-64d9-4a69-878a-41ac5d460f81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:24.152544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:24.152544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:24.152544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:24.152544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:24.152544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:24.152544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:24.152544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:24.152544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:24.152544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:24.152544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:24.152544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:24.152544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:02:24.152544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:02:24.152544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:02:24.152544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:02:24.153633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:24.153633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:24.153633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:24.153633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:24.153633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:24.153633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:24.153633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:24.153633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:24.153633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:24.153633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:24.153633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:24.153633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:24.153633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:24.153633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:24.153633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:24.153633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:24.153633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:02:24.154712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:02:24.154712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:02:24.154712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:02:24.154712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 473/960] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:02:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:24.159811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51787616-d3c4-459d-ad55-50b4a7f4e02e None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:24.253043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51787616-d3c4-459d-ad55-50b4a7f4e02e None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:24.266572 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:24.288033 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19801b-c6c2-4209-b717-fb5d1970b463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:24.288632 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19801b-c6c2-4209-b717-fb5d1970b463 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:24.288906 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19801b-c6c2-4209-b717-fb5d1970b463 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:24.289172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c19801b-c6c2-4209-b717-fb5d1970b463 None admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:24.289802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c19801b-c6c2-4209-b717-fb5d1970b463 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:24.290102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c19801b-c6c2-4209-b717-fb5d1970b463 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:24.291552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c19801b-c6c2-4209-b717-fb5d1970b463 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:24.302337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 488/961] 10.222.0.27 () {56 vars in 1109 bytes} [Mon Oct 17 03:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:24.308071 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab65929-64d9-4a69-878a-41ac5d460f81 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:24.342967 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:24.406291 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab65929-64d9-4a69-878a-41ac5d460f81 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:24.438176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88fb4a-414a-4798-9af7-47c9a177ba23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:24.438786 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88fb4a-414a-4798-9af7-47c9a177ba23 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:24.439102 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88fb4a-414a-4798-9af7-47c9a177ba23 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:24.439376 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88fb4a-414a-4798-9af7-47c9a177ba23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:24.446479 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88fb4a-414a-4798-9af7-47c9a177ba23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:24.447946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88fb4a-414a-4798-9af7-47c9a177ba23 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:24.459597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application [None req-6e88fb4a-414a-4798-9af7-47c9a177ba23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:02:24.459597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:24.459597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:24.459597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:24.459597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:24.459597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:24.459597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:24.459597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:24.459597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:24.459597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:24.459597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:24.459597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:24.459597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:02:24.459597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:02:24.459597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:02:24.460649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:02:24.460649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:24.460649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:24.460649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:24.460649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:24.460649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:24.460649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:24.460649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:24.460649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:24.460649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:24.460649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:24.460649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:24.460649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:24.460649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:24.460649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:24.460649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:24.460649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:24.461742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:02:24.461742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:02:24.461742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:02:24.461742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:02:24.461742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:02:24.461742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:02:24.461742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:02:24.461742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: ERROR keystone.server.flask.application Oct 17 03:02:24.461742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 474/962] 10.222.0.27 () {56 vars in 1111 bytes} [Mon Oct 17 03:02:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:24.466610 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19801b-c6c2-4209-b717-fb5d1970b463 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:24.562707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c19801b-c6c2-4209-b717-fb5d1970b463 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:24.595991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df93ed0-3a4e-405e-9013-fdea0e51d708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:24.596605 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df93ed0-3a4e-405e-9013-fdea0e51d708 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:24.597094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df93ed0-3a4e-405e-9013-fdea0e51d708 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:24.597414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df93ed0-3a4e-405e-9013-fdea0e51d708 None admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:24.598100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df93ed0-3a4e-405e-9013-fdea0e51d708 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:24.598407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df93ed0-3a4e-405e-9013-fdea0e51d708 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:24.599846 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df93ed0-3a4e-405e-9013-fdea0e51d708 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:24.611153 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 489/963] 10.222.0.27 () {56 vars in 1148 bytes} [Mon Oct 17 03:02:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:24.617120 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88fb4a-414a-4798-9af7-47c9a177ba23 None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:24.709311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:24.715000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88fb4a-414a-4798-9af7-47c9a177ba23 None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:24.747328 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52b029-f6c4-4d6a-99ec-76fd65b6ed52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:24.747923 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52b029-f6c4-4d6a-99ec-76fd65b6ed52 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:24.748200 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52b029-f6c4-4d6a-99ec-76fd65b6ed52 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:24.748468 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52b029-f6c4-4d6a-99ec-76fd65b6ed52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:24.749179 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52b029-f6c4-4d6a-99ec-76fd65b6ed52 None admin] RBAC: Adding query filter params (role.id=ea0fd0c10f2f4e529270edbe3525d4c6, scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2, user.id=4cc2655cc32e4d63a7000ef41bcbcfea) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:24.749484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52b029-f6c4-4d6a-99ec-76fd65b6ed52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:24.750938 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52b029-f6c4-4d6a-99ec-76fd65b6ed52 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:24.756528 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 475/964] 10.222.0.27 () {56 vars in 1374 bytes} [Mon Oct 17 03:02:24 2022] GET /identity/v3/role_assignments?user.id=4cc2655cc32e4d63a7000ef41bcbcfea&scope.project.id=89b0ee95955f465eb9ee5b8ae337d9c2&role.id=ea0fd0c10f2f4e529270edbe3525d4c6 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:26.876075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4c327-3e5a-46ed-8c41-e4a288d64e2d None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:26.876654 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4c327-3e5a-46ed-8c41-e4a288d64e2d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:26.876654 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4c327-3e5a-46ed-8c41-e4a288d64e2d None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:26.878499 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 490/965] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:02:26.883742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be461-6873-4246-8c54-a8bac0edee1d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:26.884176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be461-6873-4246-8c54-a8bac0edee1d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:26.884368 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716be461-6873-4246-8c54-a8bac0edee1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:26.903343 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-716be461-6873-4246-8c54-a8bac0edee1d None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:02:26.904527 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-716be461-6873-4246-8c54-a8bac0edee1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:02:26.974004 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 476/966] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:02:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:02:27.241609 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4c327-3e5a-46ed-8c41-e4a288d64e2d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:27.272625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:27.342969 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4c327-3e5a-46ed-8c41-e4a288d64e2d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:27.375960 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2600e5e-bdca-4683-ba89-20a4ca1bbf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:27.376580 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2600e5e-bdca-4683-ba89-20a4ca1bbf7d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:27.376861 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2600e5e-bdca-4683-ba89-20a4ca1bbf7d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:27.377132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2600e5e-bdca-4683-ba89-20a4ca1bbf7d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:27.377711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2600e5e-bdca-4683-ba89-20a4ca1bbf7d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:27.379235 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2600e5e-bdca-4683-ba89-20a4ca1bbf7d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:27.388000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-e2600e5e-bdca-4683-ba89-20a4ca1bbf7d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:02:27.388000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:27.388000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:02:27.388000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:02:27.388000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:02:27.388000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:02:27.388000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:02:27.388000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:02:27.388000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:02:27.388000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:27.388000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:02:27.388000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:27.388000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:27.388000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:27.388000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:27.388000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:27.388000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:27.389284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:27.389284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:27.389284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:27.389284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:27.389284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:27.389284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:27.389284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:02:27.389284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:02:27.389284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:02:27.389284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:02:27.389284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:27.389284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:27.389284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:27.389284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:27.389284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:27.389284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:27.390352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:27.390352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:27.390352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:27.390352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:27.390352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:27.390352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:27.390352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:27.390352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:27.390352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:27.390352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:27.390352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:02:27.390352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:02:27.390352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:02:27.390352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:27.390352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 491/967] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Oct 17 03:02:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:27.395074 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be461-6873-4246-8c54-a8bac0edee1d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:27.482412 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:27.492891 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716be461-6873-4246-8c54-a8bac0edee1d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:27.527268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d061d8-2785-4faa-8c96-531c0b108ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:27.527869 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d061d8-2785-4faa-8c96-531c0b108ade None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:27.528152 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d061d8-2785-4faa-8c96-531c0b108ade None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:27.528428 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d061d8-2785-4faa-8c96-531c0b108ade None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:27.529063 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d061d8-2785-4faa-8c96-531c0b108ade None admin] RBAC: Adding query filter params (name=placement) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:27.529369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d061d8-2785-4faa-8c96-531c0b108ade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:27.530737 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d061d8-2785-4faa-8c96-531c0b108ade None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:27.536059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 477/968] 10.222.0.27 () {56 vars in 1116 bytes} [Mon Oct 17 03:02:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:27.541792 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2600e5e-bdca-4683-ba89-20a4ca1bbf7d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:27.632638 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:27.638988 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2600e5e-bdca-4683-ba89-20a4ca1bbf7d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:27.671332 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7dccb-1b96-439e-a07e-c6eb5e2a7047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:27.671924 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7dccb-1b96-439e-a07e-c6eb5e2a7047 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:27.672199 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7dccb-1b96-439e-a07e-c6eb5e2a7047 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:27.672469 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7dccb-1b96-439e-a07e-c6eb5e2a7047 None admin] PATH_INFO: `/v3/services` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:27.673080 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7dccb-1b96-439e-a07e-c6eb5e2a7047 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:27.673378 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7dccb-1b96-439e-a07e-c6eb5e2a7047 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:27.675525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7dccb-1b96-439e-a07e-c6eb5e2a7047 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:27.679707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 492/969] 10.222.0.27 () {56 vars in 1116 bytes} [Mon Oct 17 03:02:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:29.812730 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ed5a7-e592-4076-9af2-9206564dec71 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:29.813298 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ed5a7-e592-4076-9af2-9206564dec71 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:29.813298 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ed5a7-e592-4076-9af2-9206564dec71 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:29.814597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 478/970] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:02:29.819747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169b47e-786b-4951-a6f4-b9e37d3e9154 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:29.820158 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169b47e-786b-4951-a6f4-b9e37d3e9154 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:29.820310 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169b47e-786b-4951-a6f4-b9e37d3e9154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:29.838465 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-c169b47e-786b-4951-a6f4-b9e37d3e9154 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:02:29.839696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c169b47e-786b-4951-a6f4-b9e37d3e9154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:02:29.910156 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 493/971] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:02:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:02:30.188164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ed5a7-e592-4076-9af2-9206564dec71 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:30.287208 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ed5a7-e592-4076-9af2-9206564dec71 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:30.320231 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0c8f0-4ed1-4823-ad0b-a13eb9f06259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:30.320833 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0c8f0-4ed1-4823-ad0b-a13eb9f06259 None admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:30.321110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0c8f0-4ed1-4823-ad0b-a13eb9f06259 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:30.321378 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0c8f0-4ed1-4823-ad0b-a13eb9f06259 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:30.322032 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0c8f0-4ed1-4823-ad0b-a13eb9f06259 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:30.323437 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0c8f0-4ed1-4823-ad0b-a13eb9f06259 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:30.337335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 479/972] 10.222.0.27 () {60 vars in 1141 bytes} [Mon Oct 17 03:02:30 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:02:32.462230 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5681c4e-a773-4eaf-a0e7-51f204f84835 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:32.462771 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5681c4e-a773-4eaf-a0e7-51f204f84835 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:32.462771 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5681c4e-a773-4eaf-a0e7-51f204f84835 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:32.464146 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 494/973] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:02:32.469214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ea4eb-bf80-4fcf-be0d-adf43df64a64 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:32.469638 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ea4eb-bf80-4fcf-be0d-adf43df64a64 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:32.469761 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ea4eb-bf80-4fcf-be0d-adf43df64a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:32.488099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f12ea4eb-bf80-4fcf-be0d-adf43df64a64 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:02:32.489281 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f12ea4eb-bf80-4fcf-be0d-adf43df64a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:02:32.498237 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:32.560816 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 480/974] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:02:32 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:02:32.830135 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5681c4e-a773-4eaf-a0e7-51f204f84835 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:32.930416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:32.945118 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5681c4e-a773-4eaf-a0e7-51f204f84835 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:32.979351 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9ca29-c1db-4e19-be37-8a2202d398d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:32.979973 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9ca29-c1db-4e19-be37-8a2202d398d2 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:32.980251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9ca29-c1db-4e19-be37-8a2202d398d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:32.980520 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9ca29-c1db-4e19-be37-8a2202d398d2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:32.981094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9ca29-c1db-4e19-be37-8a2202d398d2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:32.982459 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9ca29-c1db-4e19-be37-8a2202d398d2 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:32.991781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-94a9ca29-c1db-4e19-be37-8a2202d398d2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:02:32.991781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:32.991781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:02:32.991781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:02:32.991781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:02:32.991781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:02:32.991781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:02:32.991781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:02:32.991781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:02:32.991781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:32.991781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:02:32.991781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:32.991781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:32.991781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:32.991781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:32.991781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:32.991781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:32.992959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:32.992959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:32.992959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:32.992959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:32.992959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:32.992959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:32.992959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:02:32.992959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:02:32.992959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:02:32.992959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:02:32.992959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:32.992959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:32.992959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:32.992959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:32.992959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:32.992959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:32.993838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:32.993838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:32.993838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:32.993838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:32.993838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:32.993838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:32.993838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:32.993838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:32.993838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:32.993838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:32.993838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:02:32.993838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:02:32.993838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:02:32.993838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:32.993838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 495/975] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Oct 17 03:02:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:32.998941 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ea4eb-bf80-4fcf-be0d-adf43df64a64 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:33.093617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ea4eb-bf80-4fcf-be0d-adf43df64a64 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:33.125975 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113808d-9f8e-4b10-8087-cdeb05ca03bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:33.126571 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113808d-9f8e-4b10-8087-cdeb05ca03bf None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:33.126845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113808d-9f8e-4b10-8087-cdeb05ca03bf None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:33.127141 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113808d-9f8e-4b10-8087-cdeb05ca03bf None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:33.127863 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113808d-9f8e-4b10-8087-cdeb05ca03bf None admin] RBAC: Adding query filter params (name=placement) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:33.128171 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113808d-9f8e-4b10-8087-cdeb05ca03bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:33.129521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113808d-9f8e-4b10-8087-cdeb05ca03bf None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:33.134420 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 481/976] 10.222.0.27 () {56 vars in 1116 bytes} [Mon Oct 17 03:02:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:33.140641 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9ca29-c1db-4e19-be37-8a2202d398d2 None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:33.238932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9ca29-c1db-4e19-be37-8a2202d398d2 None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:33.271313 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64910c3-300f-4b50-8321-45e2810c0986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:33.271911 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64910c3-300f-4b50-8321-45e2810c0986 None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:33.272186 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64910c3-300f-4b50-8321-45e2810c0986 None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:33.272459 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64910c3-300f-4b50-8321-45e2810c0986 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:33.273148 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64910c3-300f-4b50-8321-45e2810c0986 None admin] RBAC: Adding query filter params (interface=public, service_id=f6d0cad12aa047d69d9f47e4674805ff, region_id=RegionOne) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:33.273451 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64910c3-300f-4b50-8321-45e2810c0986 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:33.274811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64910c3-300f-4b50-8321-45e2810c0986 None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:33.279815 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 496/977] 10.222.0.27 () {56 vars in 1251 bytes} [Mon Oct 17 03:02:33 2022] GET /identity/v3/endpoints?service_id=f6d0cad12aa047d69d9f47e4674805ff&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:33.286096 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113808d-9f8e-4b10-8087-cdeb05ca03bf None admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:33.289836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:33.382405 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113808d-9f8e-4b10-8087-cdeb05ca03bf None admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:33.415189 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f1d39a-d3f9-4264-9be5-d6c813f0a264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:33.415785 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1d39a-d3f9-4264-9be5-d6c813f0a264 None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:33.416064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1d39a-d3f9-4264-9be5-d6c813f0a264 None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:33.416334 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f1d39a-d3f9-4264-9be5-d6c813f0a264 None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:33.416930 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f1d39a-d3f9-4264-9be5-d6c813f0a264 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:33.418294 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f1d39a-d3f9-4264-9be5-d6c813f0a264 None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:33.425477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 482/978] 10.222.0.27 () {56 vars in 1087 bytes} [Mon Oct 17 03:02:33 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:02:35.548769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbcf56-6a2a-40f3-825a-96f73319d0cd None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:35.549289 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbcf56-6a2a-40f3-825a-96f73319d0cd None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:35.549382 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbcf56-6a2a-40f3-825a-96f73319d0cd None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:35.551354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 497/979] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:02:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:02:35.556610 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96895d60-baf7-4703-9a7e-99a23cc88a8b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:35.557040 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96895d60-baf7-4703-9a7e-99a23cc88a8b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:35.557133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96895d60-baf7-4703-9a7e-99a23cc88a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:35.575538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-96895d60-baf7-4703-9a7e-99a23cc88a8b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:02:35.576714 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-96895d60-baf7-4703-9a7e-99a23cc88a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:02:35.636511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:35.642341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 483/980] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:02:35 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:02:35.921024 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbcf56-6a2a-40f3-825a-96f73319d0cd None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:35.946709 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:36.034974 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbcf56-6a2a-40f3-825a-96f73319d0cd None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:36.069975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b350fa-6672-4a83-a8f9-57a6ae79cf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:36.070655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b350fa-6672-4a83-a8f9-57a6ae79cf0d None admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:36.070998 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b350fa-6672-4a83-a8f9-57a6ae79cf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:36.071293 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b350fa-6672-4a83-a8f9-57a6ae79cf0d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:36.071917 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b350fa-6672-4a83-a8f9-57a6ae79cf0d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:36.073342 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b350fa-6672-4a83-a8f9-57a6ae79cf0d None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:36.082993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application [None req-b3b350fa-6672-4a83-a8f9-57a6ae79cf0d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:02:36.082993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:36.082993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:02:36.082993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:02:36.082993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:02:36.082993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:02:36.082993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:02:36.082993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:02:36.082993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:02:36.082993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:36.082993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:02:36.082993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:36.082993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:02:36.082993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:02:36.082993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:02:36.082993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:02:36.082993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:02:36.084138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:02:36.084138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:02:36.084138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:02:36.084138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:02:36.084138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:02:36.084138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:02:36.084138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:02:36.084138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:02:36.084138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:02:36.084138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:02:36.084138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:02:36.084138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:02:36.084138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:02:36.084138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:02:36.084138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:02:36.084138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:02:36.084990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:02:36.084990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application with Lock( Oct 17 03:02:36.084990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:02:36.084990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:02:36.084990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:02:36.084990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:02:36.084990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:02:36.084990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:02:36.084990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:02:36.084990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:02:36.084990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:02:36.084990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:02:36.084990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:02:36.084990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: ERROR keystone.server.flask.application Oct 17 03:02:36.084990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 498/981] 10.222.0.27 () {56 vars in 1117 bytes} [Mon Oct 17 03:02:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:02:36.090062 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96895d60-baf7-4703-9a7e-99a23cc88a8b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:36.184317 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96895d60-baf7-4703-9a7e-99a23cc88a8b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:36.217575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ec8b8-a85b-4aa6-b5b9-f0fa3f219f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:36.218170 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ec8b8-a85b-4aa6-b5b9-f0fa3f219f0f None admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:36.218446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ec8b8-a85b-4aa6-b5b9-f0fa3f219f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:36.218714 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ec8b8-a85b-4aa6-b5b9-f0fa3f219f0f None admin] PATH_INFO: `/v3/services` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:36.219364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ec8b8-a85b-4aa6-b5b9-f0fa3f219f0f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:02:36.219663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ec8b8-a85b-4aa6-b5b9-f0fa3f219f0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:36.221017 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ec8b8-a85b-4aa6-b5b9-f0fa3f219f0f None admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:36.226245 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 484/982] 10.222.0.27 () {56 vars in 1116 bytes} [Mon Oct 17 03:02:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:02:36.231880 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b350fa-6672-4a83-a8f9-57a6ae79cf0d None admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:02:36.315834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:36.335232 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b350fa-6672-4a83-a8f9-57a6ae79cf0d None admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:02:36.370265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e87ab0-ac6f-4d4e-a50f-e913096f89ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:02:36.370919 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e87ab0-ac6f-4d4e-a50f-e913096f89ab None admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:02:36.371203 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e87ab0-ac6f-4d4e-a50f-e913096f89ab None admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:02:36.371478 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e87ab0-ac6f-4d4e-a50f-e913096f89ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:02:36.372139 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e87ab0-ac6f-4d4e-a50f-e913096f89ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:02:36.373526 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e87ab0-ac6f-4d4e-a50f-e913096f89ab None admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:02:36.392410 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 499/983] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:02:36 2022] POST /identity/v3/endpoints => generated 331 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:02:46.223512 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=69 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:02:46.388221 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:03:40.317435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2b532-a934-4117-94fd-8f47c5353ca6 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:40.317897 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2b532-a934-4117-94fd-8f47c5353ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:40.317984 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2b532-a934-4117-94fd-8f47c5353ca6 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:40.324958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad77807-e406-4ffb-bd03-e7529ac4832a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:40.325388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad77807-e406-4ffb-bd03-e7529ac4832a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:40.325578 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad77807-e406-4ffb-bd03-e7529ac4832a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:40.328233 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 485/984] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:03:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:03:40.411856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6ad77807-e406-4ffb-bd03-e7529ac4832a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:03:40.413071 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6ad77807-e406-4ffb-bd03-e7529ac4832a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:40.559312 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 500/985] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:03:40.840824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b64454-f3a0-485e-bece-50994431265c None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:40.841471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b64454-f3a0-485e-bece-50994431265c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:40.841953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b64454-f3a0-485e-bece-50994431265c None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:40.843962 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 486/986] 10.222.0.27 () {54 vars in 964 bytes} [Mon Oct 17 03:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:03:40.850631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac67f3d-bde8-4bb0-8d42-cfeb60e89fd5 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:40.850966 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac67f3d-bde8-4bb0-8d42-cfeb60e89fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:40.851251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac67f3d-bde8-4bb0-8d42-cfeb60e89fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:40.943127 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-bac67f3d-bde8-4bb0-8d42-cfeb60e89fd5 None None] MFA Rules not processed for user `526933e6ad004493b071f5da5caa09a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:03:40.944481 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-bac67f3d-bde8-4bb0-8d42-cfeb60e89fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:40.966535 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:03:41.037617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b64454-f3a0-485e-bece-50994431265c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:03:41.039431 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a7b64454-f3a0-485e-bece-50994431265c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:41.043981 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 501/987] 10.222.0.27 () {58 vars in 1061 bytes} [Mon Oct 17 03:03:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:03:41.172125 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b64454-f3a0-485e-bece-50994431265c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:03:41.261143 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc007d-6e7a-447d-9122-5c2876e2f95f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:03:41.261759 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc007d-6e7a-447d-9122-5c2876e2f95f service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:41.262037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc007d-6e7a-447d-9122-5c2876e2f95f service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:41.262306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc007d-6e7a-447d-9122-5c2876e2f95f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:41.264354 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b5fc007d-6e7a-447d-9122-5c2876e2f95f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:41.325020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:03:41.384251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc007d-6e7a-447d-9122-5c2876e2f95f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:03:41.385860 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc007d-6e7a-447d-9122-5c2876e2f95f service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:03:41.406693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 487/988] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:03:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:03:43.960342 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa58331-5e0a-4671-8b14-b5f79f368fc5 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:43.960768 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa58331-5e0a-4671-8b14-b5f79f368fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:43.961019 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa58331-5e0a-4671-8b14-b5f79f368fc5 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:43.962184 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 502/989] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:03:43.967475 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e6ce2-798e-4b72-9c23-a57668722537 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:43.967857 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e6ce2-798e-4b72-9c23-a57668722537 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:43.968030 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e6ce2-798e-4b72-9c23-a57668722537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:44.038157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3c7e6ce2-798e-4b72-9c23-a57668722537 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:03:44.039381 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3c7e6ce2-798e-4b72-9c23-a57668722537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:44.136801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 488/990] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:03:44.422396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa58331-5e0a-4671-8b14-b5f79f368fc5 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:03:44.424014 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ffa58331-5e0a-4671-8b14-b5f79f368fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:44.525633 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa58331-5e0a-4671-8b14-b5f79f368fc5 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:03:44.595569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:03:44.598029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9323b-0e86-4971-96f5-5a0fe8138471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:03:44.598752 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9323b-0e86-4971-96f5-5a0fe8138471 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:44.599245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9323b-0e86-4971-96f5-5a0fe8138471 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:44.599673 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9323b-0e86-4971-96f5-5a0fe8138471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:44.601444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-80c9323b-0e86-4971-96f5-5a0fe8138471 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:44.708994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9323b-0e86-4971-96f5-5a0fe8138471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:03:44.710410 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9323b-0e86-4971-96f5-5a0fe8138471 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:03:44.728324 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 503/991] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:03:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:03:47.178924 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6786dbc-16d8-4c02-bfbd-7c1676875beb None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:47.179454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6786dbc-16d8-4c02-bfbd-7c1676875beb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:47.179454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6786dbc-16d8-4c02-bfbd-7c1676875beb None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:47.180813 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 489/992] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:03:47.185878 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d527a5b-1753-40e7-9fd4-58f9b26890c8 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:47.186293 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d527a5b-1753-40e7-9fd4-58f9b26890c8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:47.186423 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d527a5b-1753-40e7-9fd4-58f9b26890c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:47.251542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3d527a5b-1753-40e7-9fd4-58f9b26890c8 None None] MFA Rules not processed for user `f5d5caad74e9468ca5bba4cd444b7404`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:03:47.252915 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3d527a5b-1753-40e7-9fd4-58f9b26890c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:47.304488 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:03:47.348284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 504/993] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:03:47 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:03:47.632426 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6786dbc-16d8-4c02-bfbd-7c1676875beb None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:03:47.633953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d6786dbc-16d8-4c02-bfbd-7c1676875beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:47.644697 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:03:47.737973 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6786dbc-16d8-4c02-bfbd-7c1676875beb None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:03:47.810641 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57980527-68c6-4d40-be07-7b20d3aacf97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:03:47.811274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57980527-68c6-4d40-be07-7b20d3aacf97 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:47.811557 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57980527-68c6-4d40-be07-7b20d3aacf97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:47.811831 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57980527-68c6-4d40-be07-7b20d3aacf97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:47.813449 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-57980527-68c6-4d40-be07-7b20d3aacf97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:47.875447 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:03:47.924538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57980527-68c6-4d40-be07-7b20d3aacf97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:03:47.926000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57980527-68c6-4d40-be07-7b20d3aacf97 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:03:47.943848 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 490/994] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:03:47 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:03:50.636499 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6e226-2e85-494a-825f-f3292461c16a None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:50.637072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6e226-2e85-494a-825f-f3292461c16a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:50.637072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6e226-2e85-494a-825f-f3292461c16a None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:50.638628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 505/995] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:03:50.644029 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2190a8a-548a-445f-93c7-10658deb5bae None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:50.644441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2190a8a-548a-445f-93c7-10658deb5bae None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:50.644594 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2190a8a-548a-445f-93c7-10658deb5bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:50.710137 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f2190a8a-548a-445f-93c7-10658deb5bae None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:03:50.711354 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f2190a8a-548a-445f-93c7-10658deb5bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:50.805341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 491/996] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:03:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:03:51.091449 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6e226-2e85-494a-825f-f3292461c16a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:03:51.092962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1be6e226-2e85-494a-825f-f3292461c16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:51.202472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6e226-2e85-494a-825f-f3292461c16a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:03:51.276366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59ae21-85e8-4ecb-a21e-69081ed84526 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:03:51.277116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59ae21-85e8-4ecb-a21e-69081ed84526 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:51.277447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59ae21-85e8-4ecb-a21e-69081ed84526 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:51.277660 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59ae21-85e8-4ecb-a21e-69081ed84526 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:51.279641 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9f59ae21-85e8-4ecb-a21e-69081ed84526 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:51.287174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:03:51.392465 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59ae21-85e8-4ecb-a21e-69081ed84526 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:03:51.393950 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59ae21-85e8-4ecb-a21e-69081ed84526 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:03:51.412312 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 506/997] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:03:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:03:54.179653 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc27c3b-b6ae-460d-b22c-0cdc20d40623 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:54.180151 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc27c3b-b6ae-460d-b22c-0cdc20d40623 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:54.180151 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc27c3b-b6ae-460d-b22c-0cdc20d40623 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:54.181441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 492/998] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:03:54.186703 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb7614-0f80-4723-a989-b4a985d22c18 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:54.187142 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb7614-0f80-4723-a989-b4a985d22c18 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:54.187286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb7614-0f80-4723-a989-b4a985d22c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:54.260015 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-73cb7614-0f80-4723-a989-b4a985d22c18 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:03:54.261270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-73cb7614-0f80-4723-a989-b4a985d22c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:54.326324 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:03:54.370149 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 507/999] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:03:54.659368 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc27c3b-b6ae-460d-b22c-0cdc20d40623 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:03:54.660993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6fc27c3b-b6ae-460d-b22c-0cdc20d40623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:54.682697 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:03:54.769706 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc27c3b-b6ae-460d-b22c-0cdc20d40623 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:03:54.841617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c4184-de53-4d80-98ac-74542480358a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:03:54.842220 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c4184-de53-4d80-98ac-74542480358a service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:54.842493 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c4184-de53-4d80-98ac-74542480358a service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:54.842759 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c4184-de53-4d80-98ac-74542480358a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:54.844364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f76c4184-de53-4d80-98ac-74542480358a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:54.918688 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:03:54.957335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c4184-de53-4d80-98ac-74542480358a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:03:54.958784 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c4184-de53-4d80-98ac-74542480358a service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:03:54.977444 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 493/1000] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:03:57.219723 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139ea85-01d6-41a1-8173-b8886a7e879d None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:57.220257 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139ea85-01d6-41a1-8173-b8886a7e879d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:57.220257 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139ea85-01d6-41a1-8173-b8886a7e879d None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:57.221657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 508/1001] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:03:57.226852 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55f5d8-40fe-4185-918c-2a7e598c698b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:57.227288 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55f5d8-40fe-4185-918c-2a7e598c698b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:57.227426 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55f5d8-40fe-4185-918c-2a7e598c698b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:57.294407 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4b55f5d8-40fe-4185-918c-2a7e598c698b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:03:57.295630 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4b55f5d8-40fe-4185-918c-2a7e598c698b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:57.402814 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 494/1002] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:03:57.683812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139ea85-01d6-41a1-8173-b8886a7e879d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:03:57.685432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1139ea85-01d6-41a1-8173-b8886a7e879d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:57.790405 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139ea85-01d6-41a1-8173-b8886a7e879d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:03:57.864274 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b251e61-c986-4e5a-88d4-26de2a1efe03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:03:57.865149 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b251e61-c986-4e5a-88d4-26de2a1efe03 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:03:57.865430 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b251e61-c986-4e5a-88d4-26de2a1efe03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:03:57.865701 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b251e61-c986-4e5a-88d4-26de2a1efe03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:03:57.867320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1b251e61-c986-4e5a-88d4-26de2a1efe03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:03:57.884142 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:03:57.981363 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b251e61-c986-4e5a-88d4-26de2a1efe03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:03:57.982715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b251e61-c986-4e5a-88d4-26de2a1efe03 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:03:58.001058 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 509/1003] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:03:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:04:00.243724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a69c4-b885-407f-ab32-44b637530346 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:00.244290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a69c4-b885-407f-ab32-44b637530346 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:00.244290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a69c4-b885-407f-ab32-44b637530346 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:00.245614 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 495/1004] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:00.251157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709f34a-5c1d-4585-838e-f69e9055b796 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:00.251615 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709f34a-5c1d-4585-838e-f69e9055b796 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:00.251705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709f34a-5c1d-4585-838e-f69e9055b796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:00.332417 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-c709f34a-5c1d-4585-838e-f69e9055b796 None None] MFA Rules not processed for user `f5d5caad74e9468ca5bba4cd444b7404`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:00.333620 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c709f34a-5c1d-4585-838e-f69e9055b796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:00.402143 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:00.436378 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 510/1005] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:04:00 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:00.718340 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a69c4-b885-407f-ab32-44b637530346 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:04:00.719904 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-453a69c4-b885-407f-ab32-44b637530346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:00.752559 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:00.827143 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a69c4-b885-407f-ab32-44b637530346 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:04:00.899886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56246f64-5545-4134-b3cc-eba1ebcd7147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:04:00.900566 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56246f64-5545-4134-b3cc-eba1ebcd7147 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:00.900867 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56246f64-5545-4134-b3cc-eba1ebcd7147 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:00.901140 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56246f64-5545-4134-b3cc-eba1ebcd7147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:00.903330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-56246f64-5545-4134-b3cc-eba1ebcd7147 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:00.983738 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:01.015088 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56246f64-5545-4134-b3cc-eba1ebcd7147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:04:01.016581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56246f64-5545-4134-b3cc-eba1ebcd7147 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:04:01.035636 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 496/1006] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:04:00 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:04:03.419852 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d452e-c947-4089-bfff-c2b032ac6cac None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:03.420427 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d452e-c947-4089-bfff-c2b032ac6cac None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:03.420427 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d452e-c947-4089-bfff-c2b032ac6cac None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:03.421742 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 511/1007] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:03.427748 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a6e15-a3a7-49d2-b42c-4ee0f8e040bf None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:03.429900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a6e15-a3a7-49d2-b42c-4ee0f8e040bf None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:03.429900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a6e15-a3a7-49d2-b42c-4ee0f8e040bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:03.493043 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ec1a6e15-a3a7-49d2-b42c-4ee0f8e040bf None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:03.494231 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ec1a6e15-a3a7-49d2-b42c-4ee0f8e040bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:03.587409 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 497/1008] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:04:03.864675 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d452e-c947-4089-bfff-c2b032ac6cac None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:04:03.866188 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d46d452e-c947-4089-bfff-c2b032ac6cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:03.967891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d452e-c947-4089-bfff-c2b032ac6cac None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:04:04.039023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a42ca5-1d72-46b6-a33e-45f9c620ed4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:04:04.039626 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a42ca5-1d72-46b6-a33e-45f9c620ed4c service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:04.039903 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a42ca5-1d72-46b6-a33e-45f9c620ed4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:04.040172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a42ca5-1d72-46b6-a33e-45f9c620ed4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:04.041721 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d8a42ca5-1d72-46b6-a33e-45f9c620ed4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:04.068691 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:04.149383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a42ca5-1d72-46b6-a33e-45f9c620ed4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:04:04.150726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a42ca5-1d72-46b6-a33e-45f9c620ed4c service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:04:04.168806 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 512/1009] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:04:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:04:06.339951 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec80ef-ab72-46bd-a107-0db3d6a5722f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:06.340464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec80ef-ab72-46bd-a107-0db3d6a5722f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:06.340464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec80ef-ab72-46bd-a107-0db3d6a5722f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:06.341818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 498/1010] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:06.347475 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc5a79-3447-47e7-ba9f-a060689006bc None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:06.347874 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc5a79-3447-47e7-ba9f-a060689006bc None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:06.348019 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc5a79-3447-47e7-ba9f-a060689006bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:06.412146 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6cbc5a79-3447-47e7-ba9f-a060689006bc None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:06.413347 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6cbc5a79-3447-47e7-ba9f-a060689006bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:06.487104 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:06.509272 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 513/1011] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:06.793808 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec80ef-ab72-46bd-a107-0db3d6a5722f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:04:06.795376 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a3ec80ef-ab72-46bd-a107-0db3d6a5722f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:06.835484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:06.897070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec80ef-ab72-46bd-a107-0db3d6a5722f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:04:06.968440 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f9ae2-c6a0-448c-8fde-26356809b09c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:04:06.969037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f9ae2-c6a0-448c-8fde-26356809b09c service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:06.969309 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f9ae2-c6a0-448c-8fde-26356809b09c service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:06.969571 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f9ae2-c6a0-448c-8fde-26356809b09c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:06.971348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8c6f9ae2-c6a0-448c-8fde-26356809b09c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:07.052709 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:07.078184 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f9ae2-c6a0-448c-8fde-26356809b09c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:04:07.079636 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6f9ae2-c6a0-448c-8fde-26356809b09c service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:04:07.097758 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 499/1012] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:04:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:04:09.578483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07760823-d63a-4b24-a2c8-e4142c49c1c8 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:09.579025 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07760823-d63a-4b24-a2c8-e4142c49c1c8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:09.579141 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07760823-d63a-4b24-a2c8-e4142c49c1c8 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:09.580503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 514/1013] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:09.585640 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84905802-f87d-4779-b5ab-ac03a699cfe8 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:09.586041 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84905802-f87d-4779-b5ab-ac03a699cfe8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:09.586193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84905802-f87d-4779-b5ab-ac03a699cfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:09.651507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-84905802-f87d-4779-b5ab-ac03a699cfe8 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:09.652705 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-84905802-f87d-4779-b5ab-ac03a699cfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:09.749177 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 500/1014] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:04:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:10.035754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07760823-d63a-4b24-a2c8-e4142c49c1c8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:04:10.037277 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-07760823-d63a-4b24-a2c8-e4142c49c1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:10.140210 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07760823-d63a-4b24-a2c8-e4142c49c1c8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:04:10.212167 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178dd34-0014-4784-ae99-c63b53ce0a28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:04:10.212750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178dd34-0014-4784-ae99-c63b53ce0a28 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:10.213031 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178dd34-0014-4784-ae99-c63b53ce0a28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:10.213303 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178dd34-0014-4784-ae99-c63b53ce0a28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:10.214928 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a178dd34-0014-4784-ae99-c63b53ce0a28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:10.251036 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:10.326273 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178dd34-0014-4784-ae99-c63b53ce0a28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:04:10.327744 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178dd34-0014-4784-ae99-c63b53ce0a28 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:04:10.346198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 515/1015] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:04:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:04:14.417073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b0215-c1b0-41aa-8b8c-95544dc142b5 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:14.417598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b0215-c1b0-41aa-8b8c-95544dc142b5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:14.417598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b0215-c1b0-41aa-8b8c-95544dc142b5 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:14.419074 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 501/1016] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:14.424901 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdefbe-488d-496f-b787-08d0366404ce None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:14.425422 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdefbe-488d-496f-b787-08d0366404ce None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:14.425533 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdefbe-488d-496f-b787-08d0366404ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:14.503818 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1efdefbe-488d-496f-b787-08d0366404ce None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:14.505040 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1efdefbe-488d-496f-b787-08d0366404ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:14.598273 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:14.610903 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 516/1017] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:04:14.916259 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711f462-f43d-40c2-81a0-8d3a19ae40bf None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:14.916546 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711f462-f43d-40c2-81a0-8d3a19ae40bf None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:14.916826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711f462-f43d-40c2-81a0-8d3a19ae40bf None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:14.918177 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 502/1018] 10.222.0.27 () {54 vars in 964 bytes} [Mon Oct 17 03:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:14.925050 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a647546-14fe-4a47-b9c0-38c7c50d035d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:14.925334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a647546-14fe-4a47-b9c0-38c7c50d035d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:14.925603 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a647546-14fe-4a47-b9c0-38c7c50d035d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:15.002775 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3a647546-14fe-4a47-b9c0-38c7c50d035d None None] MFA Rules not processed for user `526933e6ad004493b071f5da5caa09a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:15.003995 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3a647546-14fe-4a47-b9c0-38c7c50d035d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:15.080693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 517/1019] 10.222.0.27 () {58 vars in 1061 bytes} [Mon Oct 17 03:04:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:15.090728 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711f462-f43d-40c2-81a0-8d3a19ae40bf None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:04:15.092308 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6711f462-f43d-40c2-81a0-8d3a19ae40bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:15.141294 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:15.196434 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711f462-f43d-40c2-81a0-8d3a19ae40bf None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:04:15.278638 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5931dce7-55ad-4941-b357-4a8938161d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:04:15.280048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931dce7-55ad-4941-b357-4a8938161d94 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:15.280048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931dce7-55ad-4941-b357-4a8938161d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:15.280048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931dce7-55ad-4941-b357-4a8938161d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:15.281741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5931dce7-55ad-4941-b357-4a8938161d94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:15.383383 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:15.402896 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5931dce7-55ad-4941-b357-4a8938161d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:04:15.404348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5931dce7-55ad-4941-b357-4a8938161d94 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:04:15.424357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 503/1020] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:04:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:04:18.801892 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6f38b-3446-4112-abd7-dd08cc061881 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:18.802349 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6f38b-3446-4112-abd7-dd08cc061881 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:18.802458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6f38b-3446-4112-abd7-dd08cc061881 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:18.803820 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 518/1021] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:18.808895 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cacad-913e-4ee5-b89c-ca4a6e4335f7 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:18.809307 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cacad-913e-4ee5-b89c-ca4a6e4335f7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:18.809522 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cacad-913e-4ee5-b89c-ca4a6e4335f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:18.876180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d63cacad-913e-4ee5-b89c-ca4a6e4335f7 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:18.878379 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d63cacad-913e-4ee5-b89c-ca4a6e4335f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:19.002652 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 504/1022] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:04:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:19.285349 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6f38b-3446-4112-abd7-dd08cc061881 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:04:19.286894 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-38c6f38b-3446-4112-abd7-dd08cc061881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:19.374765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:19.392160 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6f38b-3446-4112-abd7-dd08cc061881 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:04:19.466152 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61386a37-6032-4637-a2d9-6b9c5a319f7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:04:19.466764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61386a37-6032-4637-a2d9-6b9c5a319f7c service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:19.467075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61386a37-6032-4637-a2d9-6b9c5a319f7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:19.467352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61386a37-6032-4637-a2d9-6b9c5a319f7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:19.468969 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-61386a37-6032-4637-a2d9-6b9c5a319f7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:19.577164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61386a37-6032-4637-a2d9-6b9c5a319f7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:04:19.578598 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61386a37-6032-4637-a2d9-6b9c5a319f7c service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:04:19.597461 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 519/1023] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:04:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:04:23.739160 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b71e9-42ef-4a24-8b2d-4032d8adcbe9 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:23.739700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b71e9-42ef-4a24-8b2d-4032d8adcbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:23.739700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b71e9-42ef-4a24-8b2d-4032d8adcbe9 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:23.741049 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 505/1024] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:23.746366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28980abd-1661-4f44-85bc-988a6ca0185f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:23.746760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28980abd-1661-4f44-85bc-988a6ca0185f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:23.747000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28980abd-1661-4f44-85bc-988a6ca0185f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:23.762775 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:23.817546 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-28980abd-1661-4f44-85bc-988a6ca0185f None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:23.818808 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-28980abd-1661-4f44-85bc-988a6ca0185f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:23.918178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 520/1025] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:24.205932 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b71e9-42ef-4a24-8b2d-4032d8adcbe9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:04:24.207833 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4d5b71e9-42ef-4a24-8b2d-4032d8adcbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:24.265583 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:24.312784 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b71e9-42ef-4a24-8b2d-4032d8adcbe9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:04:24.385849 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b9a61-d085-42c3-a01e-aa38fdd9d9e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:04:24.386387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b9a61-d085-42c3-a01e-aa38fdd9d9e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:24.386688 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b9a61-d085-42c3-a01e-aa38fdd9d9e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:24.387007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b9a61-d085-42c3-a01e-aa38fdd9d9e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:24.389118 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0c8b9a61-d085-42c3-a01e-aa38fdd9d9e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:24.494113 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:24.498279 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b9a61-d085-42c3-a01e-aa38fdd9d9e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:04:24.499756 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b9a61-d085-42c3-a01e-aa38fdd9d9e6 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:04:24.519153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 506/1026] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:04:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:04:29.456830 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c657de0-bf5c-4c2a-860a-a2317bad218e None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:29.457805 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c657de0-bf5c-4c2a-860a-a2317bad218e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:29.459730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c657de0-bf5c-4c2a-860a-a2317bad218e None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:29.464823 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 521/1027] 10.222.0.27 () {54 vars in 921 bytes} [Mon Oct 17 03:04:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:29.469971 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b9c70-d78f-44e4-8276-3a5c1ec737db None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:29.470498 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b9c70-d78f-44e4-8276-3a5c1ec737db None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:29.470792 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b9c70-d78f-44e4-8276-3a5c1ec737db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:29.616846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8f2b9c70-d78f-44e4-8276-3a5c1ec737db None None] MFA Rules not processed for user `4cc2655cc32e4d63a7000ef41bcbcfea`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:29.618163 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8f2b9c70-d78f-44e4-8276-3a5c1ec737db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:29.737953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 507/1028] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Oct 17 03:04:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:33.849896 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a1509-a020-4c79-8f26-982ee932a98c None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:33.850468 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a1509-a020-4c79-8f26-982ee932a98c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:33.850468 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a1509-a020-4c79-8f26-982ee932a98c None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:33.852005 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 522/1029] 10.222.0.27 () {54 vars in 921 bytes} [Mon Oct 17 03:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:33.857035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506064a8-4f56-4fa3-879f-25cea8b883d5 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:33.857827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506064a8-4f56-4fa3-879f-25cea8b883d5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:33.858364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506064a8-4f56-4fa3-879f-25cea8b883d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:33.916574 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=59 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:33.919331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:33.945456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-506064a8-4f56-4fa3-879f-25cea8b883d5 None None] MFA Rules not processed for user `4cc2655cc32e4d63a7000ef41bcbcfea`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:33.947084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-506064a8-4f56-4fa3-879f-25cea8b883d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:34.060377 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 508/1030] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Oct 17 03:04:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:35.377936 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea941-3f05-4ba0-a9c7-ed0aae540825 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:35.378486 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea941-3f05-4ba0-a9c7-ed0aae540825 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:35.378486 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea941-3f05-4ba0-a9c7-ed0aae540825 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:35.379839 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 523/1031] 10.222.0.27 () {54 vars in 921 bytes} [Mon Oct 17 03:04:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:35.384790 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ec721-19cd-4278-9d92-28b816d5d8de None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:35.385195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ec721-19cd-4278-9d92-28b816d5d8de None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:35.385347 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263ec721-19cd-4278-9d92-28b816d5d8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:35.462766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-263ec721-19cd-4278-9d92-28b816d5d8de None None] MFA Rules not processed for user `4cc2655cc32e4d63a7000ef41bcbcfea`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:35.464000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-263ec721-19cd-4278-9d92-28b816d5d8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:35.506357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:35.571376 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 509/1032] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Oct 17 03:04:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:04:39.259319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dad8c-65c4-4250-aca8-53e1275b9b66 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:39.260141 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dad8c-65c4-4250-aca8-53e1275b9b66 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:39.260417 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00dad8c-65c4-4250-aca8-53e1275b9b66 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:39.266120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 524/1033] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:04:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:39.268165 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd069f-5206-4c65-81ec-af043ce78917 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:39.268638 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd069f-5206-4c65-81ec-af043ce78917 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:39.271573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd069f-5206-4c65-81ec-af043ce78917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:39.356036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-49dd069f-5206-4c65-81ec-af043ce78917 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:39.357265 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-49dd069f-5206-4c65-81ec-af043ce78917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:39.460113 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 510/1034] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:39.767565 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f6f8a-7367-4952-a1ff-164334f0b0d3 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:39.768056 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f6f8a-7367-4952-a1ff-164334f0b0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:39.768490 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f6f8a-7367-4952-a1ff-164334f0b0d3 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:39.769827 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 525/1035] 10.222.0.27 () {54 vars in 961 bytes} [Mon Oct 17 03:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:39.776827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0130f-e2b4-4b4d-8219-3fa439757ae2 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:39.777326 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0130f-e2b4-4b4d-8219-3fa439757ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:39.777766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0130f-e2b4-4b4d-8219-3fa439757ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:39.798734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:39.868179 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-0fd0130f-e2b4-4b4d-8219-3fa439757ae2 None None] MFA Rules not processed for user `01637fc07ba846aebcb42325bed46216`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:39.869391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0fd0130f-e2b4-4b4d-8219-3fa439757ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:40.009583 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 511/1036] 10.222.0.27 () {58 vars in 1058 bytes} [Mon Oct 17 03:04:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:40.018003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f6f8a-7367-4952-a1ff-164334f0b0d3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:04:40.019646 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1b3f6f8a-7367-4952-a1ff-164334f0b0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:40.180012 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f6f8a-7367-4952-a1ff-164334f0b0d3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:04:40.258010 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:40.301010 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c331b-5b85-4203-b254-9671ec5f55b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:04:40.301710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c331b-5b85-4203-b254-9671ec5f55b5 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:40.301978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c331b-5b85-4203-b254-9671ec5f55b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:40.302180 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c331b-5b85-4203-b254-9671ec5f55b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:40.303921 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-484c331b-5b85-4203-b254-9671ec5f55b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:40.403942 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c331b-5b85-4203-b254-9671ec5f55b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:04:40.405691 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c331b-5b85-4203-b254-9671ec5f55b5 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:04:40.426406 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 526/1037] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:04:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:04:42.812603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa71641-e0af-4702-980c-44040d894601 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:42.812603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa71641-e0af-4702-980c-44040d894601 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:42.812603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa71641-e0af-4702-980c-44040d894601 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:42.813330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 512/1038] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:42.818459 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d702ee-5d01-4d5f-a867-69fa53c96332 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:42.818931 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d702ee-5d01-4d5f-a867-69fa53c96332 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:42.819054 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d702ee-5d01-4d5f-a867-69fa53c96332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:42.887506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e1d702ee-5d01-4d5f-a867-69fa53c96332 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:42.888876 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e1d702ee-5d01-4d5f-a867-69fa53c96332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:42.930661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:42.997278 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 527/1039] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:43.291478 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa71641-e0af-4702-980c-44040d894601 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:04:43.292965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cfa71641-e0af-4702-980c-44040d894601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:43.364401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:43.475816 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa71641-e0af-4702-980c-44040d894601 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:04:43.606159 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bfe19-269c-4076-95f6-c409f59c7b92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:04:43.606781 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bfe19-269c-4076-95f6-c409f59c7b92 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:43.607094 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bfe19-269c-4076-95f6-c409f59c7b92 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:43.607366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bfe19-269c-4076-95f6-c409f59c7b92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:43.608966 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-246bfe19-269c-4076-95f6-c409f59c7b92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:43.633650 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:43.704392 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bfe19-269c-4076-95f6-c409f59c7b92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:04:43.706553 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bfe19-269c-4076-95f6-c409f59c7b92 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:04:43.725476 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 513/1040] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:04:46.240807 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff952cf-9751-488b-9d64-bdf9b35482d1 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:46.241760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff952cf-9751-488b-9d64-bdf9b35482d1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:46.242074 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff952cf-9751-488b-9d64-bdf9b35482d1 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:46.243468 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 528/1041] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:04:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:04:46.248733 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec99e9-8eb1-47a6-b46a-4f74d9de67b6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:46.249156 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec99e9-8eb1-47a6-b46a-4f74d9de67b6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:46.249295 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec99e9-8eb1-47a6-b46a-4f74d9de67b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:46.323172 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-12ec99e9-8eb1-47a6-b46a-4f74d9de67b6 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:46.324412 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-12ec99e9-8eb1-47a6-b46a-4f74d9de67b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:46.430150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:46.439095 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 514/1042] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:46.724548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c7a4c-7022-4426-98c6-b3b26b32f3e3 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:46.724955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c7a4c-7022-4426-98c6-b3b26b32f3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:46.725330 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c7a4c-7022-4426-98c6-b3b26b32f3e3 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:46.726772 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 529/1043] 10.222.0.27 () {54 vars in 961 bytes} [Mon Oct 17 03:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:46.732160 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68fe4d-1b0f-484c-85fd-62aa0044dec1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:46.732528 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68fe4d-1b0f-484c-85fd-62aa0044dec1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:46.732855 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68fe4d-1b0f-484c-85fd-62aa0044dec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:46.812366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5d68fe4d-1b0f-484c-85fd-62aa0044dec1 None None] MFA Rules not processed for user `01637fc07ba846aebcb42325bed46216`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:46.813555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5d68fe4d-1b0f-484c-85fd-62aa0044dec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:46.935899 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 515/1044] 10.222.0.27 () {58 vars in 1058 bytes} [Mon Oct 17 03:04:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:46.943929 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c7a4c-7022-4426-98c6-b3b26b32f3e3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:04:46.945472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-957c7a4c-7022-4426-98c6-b3b26b32f3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:47.105420 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c7a4c-7022-4426-98c6-b3b26b32f3e3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:04:47.127297 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:47.228675 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602b01d-5689-482f-a5a8-b976efe62630 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:04:47.229465 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602b01d-5689-482f-a5a8-b976efe62630 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:47.229872 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602b01d-5689-482f-a5a8-b976efe62630 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:47.230380 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602b01d-5689-482f-a5a8-b976efe62630 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:47.232124 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3602b01d-5689-482f-a5a8-b976efe62630 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:47.342213 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602b01d-5689-482f-a5a8-b976efe62630 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:04:47.344216 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602b01d-5689-482f-a5a8-b976efe62630 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:04:47.365063 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 530/1045] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:04:47.420752 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c5341-91ea-451a-9c71-4fc418a8f4c5 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:47.421239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c5341-91ea-451a-9c71-4fc418a8f4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:47.421409 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c5341-91ea-451a-9c71-4fc418a8f4c5 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:47.422812 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 516/1046] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:04:47.428765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb08546-888d-4f8f-8947-0f8c11f03fe5 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:47.429057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb08546-888d-4f8f-8947-0f8c11f03fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:47.429337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb08546-888d-4f8f-8947-0f8c11f03fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:47.465919 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:47.516943 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2eb08546-888d-4f8f-8947-0f8c11f03fe5 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:47.519122 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2eb08546-888d-4f8f-8947-0f8c11f03fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:47.638760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 531/1047] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:47.974118 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42e7c6-783b-452b-97eb-59eff2e95452 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:47.974662 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42e7c6-783b-452b-97eb-59eff2e95452 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:47.975242 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42e7c6-783b-452b-97eb-59eff2e95452 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:47.977062 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 517/1048] 10.222.0.27 () {54 vars in 963 bytes} [Mon Oct 17 03:04:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:47.982070 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bafebd-f083-41ac-acbc-e3484aaa55ed None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:47.982070 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bafebd-f083-41ac-acbc-e3484aaa55ed None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:47.982260 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bafebd-f083-41ac-acbc-e3484aaa55ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:48.062353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f5bafebd-f083-41ac-acbc-e3484aaa55ed None None] MFA Rules not processed for user `67abf21b99d349dfa7d88a0e18be1fed`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:48.063578 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f5bafebd-f083-41ac-acbc-e3484aaa55ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:48.118397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:48.152065 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 532/1049] 10.222.0.27 () {58 vars in 1060 bytes} [Mon Oct 17 03:04:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:48.160303 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42e7c6-783b-452b-97eb-59eff2e95452 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:04:48.161922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8d42e7c6-783b-452b-97eb-59eff2e95452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:48.220606 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:48.284381 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42e7c6-783b-452b-97eb-59eff2e95452 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:04:48.370570 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc09e5-a03a-4af2-8349-2c0f02a0a4d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:04:48.371451 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc09e5-a03a-4af2-8349-2c0f02a0a4d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:48.371858 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc09e5-a03a-4af2-8349-2c0f02a0a4d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:48.372141 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc09e5-a03a-4af2-8349-2c0f02a0a4d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:48.374021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-acbc09e5-a03a-4af2-8349-2c0f02a0a4d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:48.493895 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:48.511233 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc09e5-a03a-4af2-8349-2c0f02a0a4d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:04:48.512837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc09e5-a03a-4af2-8349-2c0f02a0a4d1 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:04:48.534391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 518/1050] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:04:49.639944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b1713-47c0-42be-a8bd-2836ac665b17 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:49.640513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b1713-47c0-42be-a8bd-2836ac665b17 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:49.640513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9b1713-47c0-42be-a8bd-2836ac665b17 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:49.641809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 533/1051] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:49.647091 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f039213-d4c3-4be2-b3ff-2ff70cc78906 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:49.647519 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f039213-d4c3-4be2-b3ff-2ff70cc78906 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:49.647771 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f039213-d4c3-4be2-b3ff-2ff70cc78906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:49.714492 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4f039213-d4c3-4be2-b3ff-2ff70cc78906 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:49.715712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4f039213-d4c3-4be2-b3ff-2ff70cc78906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:49.811511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 519/1052] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:50.107765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b1713-47c0-42be-a8bd-2836ac665b17 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:04:50.109459 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ef9b1713-47c0-42be-a8bd-2836ac665b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:50.265236 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9b1713-47c0-42be-a8bd-2836ac665b17 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:04:50.322390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:50.385996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b34145-295a-4a25-91a0-7bee2fb7e0fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:04:50.386611 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b34145-295a-4a25-91a0-7bee2fb7e0fc service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:50.386936 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b34145-295a-4a25-91a0-7bee2fb7e0fc service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:50.387617 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b34145-295a-4a25-91a0-7bee2fb7e0fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:50.388755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d9b34145-295a-4a25-91a0-7bee2fb7e0fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:50.482066 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b34145-295a-4a25-91a0-7bee2fb7e0fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:04:50.483672 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b34145-295a-4a25-91a0-7bee2fb7e0fc service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:04:50.502907 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 534/1053] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:04:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:04:52.749741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca581f-69de-4797-9df1-a562b2273204 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:52.750270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca581f-69de-4797-9df1-a562b2273204 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:52.750270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca581f-69de-4797-9df1-a562b2273204 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:52.752543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 520/1054] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:04:52.758003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72129f-2009-4c89-a5bc-b529d9225fb7 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:52.758431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72129f-2009-4c89-a5bc-b529d9225fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:52.758554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72129f-2009-4c89-a5bc-b529d9225fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:52.826484 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-cf72129f-2009-4c89-a5bc-b529d9225fb7 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:52.827820 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cf72129f-2009-4c89-a5bc-b529d9225fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:52.837012 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:52.946796 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 535/1055] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:53.237220 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca581f-69de-4797-9df1-a562b2273204 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:04:53.238893 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-91ca581f-69de-4797-9df1-a562b2273204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:53.309048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:53.397334 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca581f-69de-4797-9df1-a562b2273204 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:04:53.514781 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778126c5-abb3-4dd5-9d14-82ff8451b4e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:04:53.515325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778126c5-abb3-4dd5-9d14-82ff8451b4e1 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:53.515603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778126c5-abb3-4dd5-9d14-82ff8451b4e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:53.515872 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778126c5-abb3-4dd5-9d14-82ff8451b4e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:53.518095 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-778126c5-abb3-4dd5-9d14-82ff8451b4e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:53.538626 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:53.612490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778126c5-abb3-4dd5-9d14-82ff8451b4e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:04:53.614109 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778126c5-abb3-4dd5-9d14-82ff8451b4e1 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:04:53.632959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 521/1056] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:04:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:04:55.845533 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00573266-28dc-4be3-a7b7-9d923a072bcb None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:55.846088 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00573266-28dc-4be3-a7b7-9d923a072bcb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:55.846406 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00573266-28dc-4be3-a7b7-9d923a072bcb None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:55.848074 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 536/1057] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:04:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:55.853460 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2a489-8780-4a59-87f5-d8c4f864f59b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:55.853921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2a489-8780-4a59-87f5-d8c4f864f59b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:55.854003 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a2a489-8780-4a59-87f5-d8c4f864f59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:55.927447 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c2a2a489-8780-4a59-87f5-d8c4f864f59b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:55.928730 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c2a2a489-8780-4a59-87f5-d8c4f864f59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:56.033689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:56.059716 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 522/1058] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:04:56.355804 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00573266-28dc-4be3-a7b7-9d923a072bcb None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:04:56.357393 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-00573266-28dc-4be3-a7b7-9d923a072bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:56.500384 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:56.512872 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00573266-28dc-4be3-a7b7-9d923a072bcb None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:04:56.637246 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078956a-80f5-4fa8-b3df-593c37cddebe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:04:56.637878 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078956a-80f5-4fa8-b3df-593c37cddebe service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:56.638174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078956a-80f5-4fa8-b3df-593c37cddebe service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:56.638463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078956a-80f5-4fa8-b3df-593c37cddebe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:56.640417 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4078956a-80f5-4fa8-b3df-593c37cddebe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:56.734248 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078956a-80f5-4fa8-b3df-593c37cddebe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:04:56.736476 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078956a-80f5-4fa8-b3df-593c37cddebe service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:04:56.746344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:04:56.755645 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 537/1059] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:04:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:04:59.521977 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bf14f-1aec-426b-8c04-924dd1e25f85 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:59.522488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bf14f-1aec-426b-8c04-924dd1e25f85 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:59.522488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bf14f-1aec-426b-8c04-924dd1e25f85 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:59.523863 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 523/1060] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:04:59.529373 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1ce5-3daf-46fa-8721-0d60dc2b9b90 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:04:59.529786 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1ce5-3daf-46fa-8721-0d60dc2b9b90 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:04:59.529960 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e1ce5-3daf-46fa-8721-0d60dc2b9b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:04:59.599814 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d40e1ce5-3daf-46fa-8721-0d60dc2b9b90 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:04:59.601034 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d40e1ce5-3daf-46fa-8721-0d60dc2b9b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:04:59.710899 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 538/1061] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:00.006480 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bf14f-1aec-426b-8c04-924dd1e25f85 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:00.008669 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ef1bf14f-1aec-426b-8c04-924dd1e25f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:00.162686 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bf14f-1aec-426b-8c04-924dd1e25f85 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:00.232743 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:00.287395 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a0967-e419-485f-ba8d-3258779abd87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:00.288027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a0967-e419-485f-ba8d-3258779abd87 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:00.288289 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a0967-e419-485f-ba8d-3258779abd87 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:00.288560 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a0967-e419-485f-ba8d-3258779abd87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:00.290157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-057a0967-e419-485f-ba8d-3258779abd87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:00.389051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a0967-e419-485f-ba8d-3258779abd87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:00.391035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a0967-e419-485f-ba8d-3258779abd87 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:00.410696 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 524/1062] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:03.002890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba64725-d146-4159-898a-35f49d8e8830 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:03.003472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba64725-d146-4159-898a-35f49d8e8830 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:03.003472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba64725-d146-4159-898a-35f49d8e8830 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:03.004801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 539/1063] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:03.009977 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff759d-d013-499c-9f02-fbd39fc6f7f7 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:03.010382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff759d-d013-499c-9f02-fbd39fc6f7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:03.010527 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff759d-d013-499c-9f02-fbd39fc6f7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:03.076248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-87ff759d-d013-499c-9f02-fbd39fc6f7f7 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:03.077454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-87ff759d-d013-499c-9f02-fbd39fc6f7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:03.111283 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:03.176500 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 525/1064] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:03.465816 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba64725-d146-4159-898a-35f49d8e8830 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:03.467337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fba64725-d146-4159-898a-35f49d8e8830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:03.544073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:03.615637 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba64725-d146-4159-898a-35f49d8e8830 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:03.730087 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1768ab9-acd3-4c1e-b9e3-9e2cb7226055 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:03.730725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1768ab9-acd3-4c1e-b9e3-9e2cb7226055 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:03.731050 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1768ab9-acd3-4c1e-b9e3-9e2cb7226055 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:03.731322 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1768ab9-acd3-4c1e-b9e3-9e2cb7226055 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:03.732903 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f1768ab9-acd3-4c1e-b9e3-9e2cb7226055 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:03.772843 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:03.823981 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1768ab9-acd3-4c1e-b9e3-9e2cb7226055 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:03.825486 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1768ab9-acd3-4c1e-b9e3-9e2cb7226055 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:03.843577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 540/1065] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:04.121989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc70ff-d8fc-4c5f-9531-c50136673c74 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:04.122542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc70ff-d8fc-4c5f-9531-c50136673c74 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:04.122542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc70ff-d8fc-4c5f-9531-c50136673c74 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:04.123965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 526/1066] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:04.129576 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461bdd0-227f-46c6-9de8-ddd9b282ec91 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:04.130079 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461bdd0-227f-46c6-9de8-ddd9b282ec91 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:04.130161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461bdd0-227f-46c6-9de8-ddd9b282ec91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:04.201482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7461bdd0-227f-46c6-9de8-ddd9b282ec91 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:04.202694 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7461bdd0-227f-46c6-9de8-ddd9b282ec91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:04.313350 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 541/1067] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:04.617537 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce1b8c-584c-42cf-9f8f-5271dd6933d1 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:04.617821 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce1b8c-584c-42cf-9f8f-5271dd6933d1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:04.618095 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce1b8c-584c-42cf-9f8f-5271dd6933d1 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:04.619465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 527/1068] 10.222.0.27 () {54 vars in 962 bytes} [Mon Oct 17 03:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:04.624783 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defc703-ca35-4205-8ad0-3c778154795a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:04.625103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defc703-ca35-4205-8ad0-3c778154795a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:04.625379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defc703-ca35-4205-8ad0-3c778154795a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:04.641617 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:04.703046 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2defc703-ca35-4205-8ad0-3c778154795a None None] MFA Rules not processed for user `a8aa29639a3143d2a270e3248d2205d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:04.704775 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2defc703-ca35-4205-8ad0-3c778154795a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:04.783021 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 542/1069] 10.222.0.27 () {58 vars in 1059 bytes} [Mon Oct 17 03:05:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:05:04.791994 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce1b8c-584c-42cf-9f8f-5271dd6933d1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:04.793480 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b7ce1b8c-584c-42cf-9f8f-5271dd6933d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:04.927729 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce1b8c-584c-42cf-9f8f-5271dd6933d1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:04.986997 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:05.003425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418abdc-ee82-4612-b0ce-a769b4cee951 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8aa29639a3143d2a270e3248d2205d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:05.004034 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418abdc-ee82-4612-b0ce-a769b4cee951 service glance] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:05.004317 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418abdc-ee82-4612-b0ce-a769b4cee951 service glance] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:05.004598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418abdc-ee82-4612-b0ce-a769b4cee951 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:05.006193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7418abdc-ee82-4612-b0ce-a769b4cee951 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:05.118443 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418abdc-ee82-4612-b0ce-a769b4cee951 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:05.119922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418abdc-ee82-4612-b0ce-a769b4cee951 service glance] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:05.138636 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 528/1070] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:05.908875 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cfa1e-620c-4e0d-afae-bc8740d8fac2 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:05.909394 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cfa1e-620c-4e0d-afae-bc8740d8fac2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:05.909500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cfa1e-620c-4e0d-afae-bc8740d8fac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:05.984062 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8d3cfa1e-620c-4e0d-afae-bc8740d8fac2 None None] MFA Rules not processed for user `280d47ed79f14105a83b18b2cac58b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:05.985282 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8d3cfa1e-620c-4e0d-afae-bc8740d8fac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:06.040737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:06.067689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4e421-0eb7-4f8f-88d3-13feb79e7486 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:06.067982 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4e421-0eb7-4f8f-88d3-13feb79e7486 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:06.068252 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4e421-0eb7-4f8f-88d3-13feb79e7486 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:06.069945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 529/1071] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:05:06.073260 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 543/1072] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Oct 17 03:05:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:06.074941 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124a33a-9190-401a-84fc-bfcd9d28fe54 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:06.075268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124a33a-9190-401a-84fc-bfcd9d28fe54 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:06.075615 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124a33a-9190-401a-84fc-bfcd9d28fe54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:06.119809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cd5c4-12f1-4407-a607-771c6838a7b9 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:06.120177 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cd5c4-12f1-4407-a607-771c6838a7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:06.120573 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cd5c4-12f1-4407-a607-771c6838a7b9 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:06.121974 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 544/1073] 10.222.0.27 () {54 vars in 943 bytes} [Mon Oct 17 03:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:05:06.127990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778a270-0a3e-4749-aa07-a55fe2d11afb None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:06.128279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778a270-0a3e-4749-aa07-a55fe2d11afb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:06.128554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778a270-0a3e-4749-aa07-a55fe2d11afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:06.146608 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d124a33a-9190-401a-84fc-bfcd9d28fe54 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:06.147860 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d124a33a-9190-401a-84fc-bfcd9d28fe54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:06.190028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:06.207915 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7778a270-0a3e-4749-aa07-a55fe2d11afb None None] MFA Rules not processed for user `eceda8b3a8ba4233851d0138d8f4d06a`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:06.209179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7778a270-0a3e-4749-aa07-a55fe2d11afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:06.254576 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 530/1074] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:06.320314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 545/1075] 10.222.0.27 () {58 vars in 1040 bytes} [Mon Oct 17 03:05:06 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:06.329848 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124a33a-9190-401a-84fc-bfcd9d28fe54 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:06.331099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d124a33a-9190-401a-84fc-bfcd9d28fe54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:06.433186 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124a33a-9190-401a-84fc-bfcd9d28fe54 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:06.494756 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:06.536038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43d745-9eb2-4826-9791-35192cacac5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eceda8b3a8ba4233851d0138d8f4d06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:06.536657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43d745-9eb2-4826-9791-35192cacac5f service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:06.536934 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43d745-9eb2-4826-9791-35192cacac5f service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:06.537204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43d745-9eb2-4826-9791-35192cacac5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:06.538839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6a43d745-9eb2-4826-9791-35192cacac5f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:06.542787 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778a270-0a3e-4749-aa07-a55fe2d11afb None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:06.544102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7778a270-0a3e-4749-aa07-a55fe2d11afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:06.576747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:06.623928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43d745-9eb2-4826-9791-35192cacac5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:06.625610 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43d745-9eb2-4826-9791-35192cacac5f service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:06.634149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 531/1076] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Oct 17 03:05:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:05:06.646822 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778a270-0a3e-4749-aa07-a55fe2d11afb None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:06.781660 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4b30c-1f8e-4a00-9fab-58209462a51a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:06.782269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4b30c-1f8e-4a00-9fab-58209462a51a service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:06.783232 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4b30c-1f8e-4a00-9fab-58209462a51a service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:06.783549 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4b30c-1f8e-4a00-9fab-58209462a51a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:06.785169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5cc4b30c-1f8e-4a00-9fab-58209462a51a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:06.831299 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:06.884524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4b30c-1f8e-4a00-9fab-58209462a51a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:06.886104 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4b30c-1f8e-4a00-9fab-58209462a51a service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:06.907798 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 546/1077] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:05:09.121168 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18147470-6896-4156-8d7b-57cc05ea57af None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:09.121726 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18147470-6896-4156-8d7b-57cc05ea57af None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:09.121726 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18147470-6896-4156-8d7b-57cc05ea57af None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:09.123259 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 532/1078] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:09.128764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda26992-6492-4977-a001-6e1c30f8375f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:09.129267 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda26992-6492-4977-a001-6e1c30f8375f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:09.129374 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda26992-6492-4977-a001-6e1c30f8375f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:09.202986 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-dda26992-6492-4977-a001-6e1c30f8375f None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:09.204733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dda26992-6492-4977-a001-6e1c30f8375f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:09.319955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 547/1079] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:09.609672 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18147470-6896-4156-8d7b-57cc05ea57af None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:09.612029 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-18147470-6896-4156-8d7b-57cc05ea57af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:09.723515 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:09.781550 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18147470-6896-4156-8d7b-57cc05ea57af None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:09.940785 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fbaa1-e781-4655-8b22-7b262eb88847 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:09.941750 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fbaa1-e781-4655-8b22-7b262eb88847 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:09.945117 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fbaa1-e781-4655-8b22-7b262eb88847 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:09.945469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fbaa1-e781-4655-8b22-7b262eb88847 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:09.949473 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-723fbaa1-e781-4655-8b22-7b262eb88847 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:10.098494 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:10.135900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723fbaa1-e781-4655-8b22-7b262eb88847 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:10.137601 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723fbaa1-e781-4655-8b22-7b262eb88847 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:10.159904 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 533/1080] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:12.421628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e62426-5b2e-4741-8b67-40b4d91f7b24 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:12.422205 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e62426-5b2e-4741-8b67-40b4d91f7b24 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:12.422205 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e62426-5b2e-4741-8b67-40b4d91f7b24 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:12.424161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 548/1081] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:12.429567 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2d861-7fb1-4730-9350-354fd69103e1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:12.430039 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2d861-7fb1-4730-9350-354fd69103e1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:12.430147 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2d861-7fb1-4730-9350-354fd69103e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:12.508320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-90a2d861-7fb1-4730-9350-354fd69103e1 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:12.509831 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-90a2d861-7fb1-4730-9350-354fd69103e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:12.631174 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 534/1082] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:12.918309 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e62426-5b2e-4741-8b67-40b4d91f7b24 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:12.919857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-41e62426-5b2e-4741-8b67-40b4d91f7b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:12.941380 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:13.096592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e62426-5b2e-4741-8b67-40b4d91f7b24 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:13.194696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:13.223417 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ee2fe-0801-42e1-8093-fac1b3fa2bf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:13.224081 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ee2fe-0801-42e1-8093-fac1b3fa2bf6 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:13.224363 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ee2fe-0801-42e1-8093-fac1b3fa2bf6 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:13.224625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ee2fe-0801-42e1-8093-fac1b3fa2bf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:13.226226 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-af0ee2fe-0801-42e1-8093-fac1b3fa2bf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:13.332251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ee2fe-0801-42e1-8093-fac1b3fa2bf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:13.334199 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ee2fe-0801-42e1-8093-fac1b3fa2bf6 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:13.354678 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 549/1083] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:15.584841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f73bea-b0b7-4fb4-8b5a-82881a00a9da None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:15.585433 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f73bea-b0b7-4fb4-8b5a-82881a00a9da None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:15.585433 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f73bea-b0b7-4fb4-8b5a-82881a00a9da None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:15.588105 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 535/1084] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:05:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:15.592455 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6da73e-e6da-41cb-9faf-0986dd3afe4c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:15.592920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6da73e-e6da-41cb-9faf-0986dd3afe4c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:15.593021 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6da73e-e6da-41cb-9faf-0986dd3afe4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:15.670744 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8f6da73e-e6da-41cb-9faf-0986dd3afe4c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:15.672150 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8f6da73e-e6da-41cb-9faf-0986dd3afe4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:15.741601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:15.798546 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 550/1085] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:05:16.097905 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f73bea-b0b7-4fb4-8b5a-82881a00a9da None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:16.099783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-85f73bea-b0b7-4fb4-8b5a-82881a00a9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:16.150456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:16.269110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f73bea-b0b7-4fb4-8b5a-82881a00a9da None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:16.398477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfde48b-ed17-487f-a24b-f63e881000db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:16.399112 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde48b-ed17-487f-a24b-f63e881000db service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:16.399390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde48b-ed17-487f-a24b-f63e881000db service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:16.399657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde48b-ed17-487f-a24b-f63e881000db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:16.401330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7dfde48b-ed17-487f-a24b-f63e881000db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:16.403554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:16.505590 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfde48b-ed17-487f-a24b-f63e881000db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:16.507340 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfde48b-ed17-487f-a24b-f63e881000db service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:16.527857 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 536/1086] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:25.796352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=19 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:26.525642 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=44 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:29.544192 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb636430-381f-429e-9331-598ad02f9d9e None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:29.544754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb636430-381f-429e-9331-598ad02f9d9e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:29.544849 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb636430-381f-429e-9331-598ad02f9d9e None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:29.546163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 551/1087] 10.222.0.27 () {54 vars in 918 bytes} [Mon Oct 17 03:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:29.551344 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab2a5-3d6c-4e80-b434-cb3167dcb18f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:29.551784 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab2a5-3d6c-4e80-b434-cb3167dcb18f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:29.551907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ab2a5-3d6c-4e80-b434-cb3167dcb18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:29.628148 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-813ab2a5-3d6c-4e80-b434-cb3167dcb18f None None] MFA Rules not processed for user `4cc2655cc32e4d63a7000ef41bcbcfea`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:29.629363 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-813ab2a5-3d6c-4e80-b434-cb3167dcb18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:29.736362 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 537/1088] 10.222.0.27 () {58 vars in 1015 bytes} [Mon Oct 17 03:05:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:29.750383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993a8ff-ed70-4b38-a7b8-4a302fdad572 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:29.750688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993a8ff-ed70-4b38-a7b8-4a302fdad572 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:29.751041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993a8ff-ed70-4b38-a7b8-4a302fdad572 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:29.752359 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 552/1089] 10.222.0.27 () {54 vars in 918 bytes} [Mon Oct 17 03:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:29.757208 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca3b1f-e738-4167-82df-4a942a8797c8 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:29.757491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca3b1f-e738-4167-82df-4a942a8797c8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:29.757764 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca3b1f-e738-4167-82df-4a942a8797c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:29.830707 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:29.833766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-72ca3b1f-e738-4167-82df-4a942a8797c8 None None] MFA Rules not processed for user `01637fc07ba846aebcb42325bed46216`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:29.835610 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-72ca3b1f-e738-4167-82df-4a942a8797c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:29.949505 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 538/1090] 10.222.0.27 () {58 vars in 1015 bytes} [Mon Oct 17 03:05:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:05:32.513343 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b3d8c-f51f-4f03-a364-a84efa8264e4 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:32.513872 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b3d8c-f51f-4f03-a364-a84efa8264e4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:32.513872 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b3d8c-f51f-4f03-a364-a84efa8264e4 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:32.515222 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 553/1091] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:32.521193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3274be-a9a5-4214-b114-fabdbc58fa42 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:32.521635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3274be-a9a5-4214-b114-fabdbc58fa42 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:32.521766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3274be-a9a5-4214-b114-fabdbc58fa42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:32.586768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2d3274be-a9a5-4214-b114-fabdbc58fa42 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:32.587983 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2d3274be-a9a5-4214-b114-fabdbc58fa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:32.642482 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:32.684944 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 539/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:32.974177 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b3d8c-f51f-4f03-a364-a84efa8264e4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:32.975787 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5d9b3d8c-f51f-4f03-a364-a84efa8264e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:33.120403 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b3d8c-f51f-4f03-a364-a84efa8264e4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:33.192271 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:33.232873 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5eb0f-3f84-48c4-b898-77edfac40522 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:33.233406 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5eb0f-3f84-48c4-b898-77edfac40522 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:33.233683 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5eb0f-3f84-48c4-b898-77edfac40522 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:33.233955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5eb0f-3f84-48c4-b898-77edfac40522 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:33.235573 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-75a5eb0f-3f84-48c4-b898-77edfac40522 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:33.324996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5eb0f-3f84-48c4-b898-77edfac40522 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:33.326579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5eb0f-3f84-48c4-b898-77edfac40522 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:33.344658 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 554/1093] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:34.479244 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad008d5-9afa-47a9-8f62-52dad73148b3 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:34.479801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad008d5-9afa-47a9-8f62-52dad73148b3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:34.479801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad008d5-9afa-47a9-8f62-52dad73148b3 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:34.481206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 540/1094] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:34.487408 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d3ef7-52ed-4838-a0b0-95ccafa7c617 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:34.487849 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d3ef7-52ed-4838-a0b0-95ccafa7c617 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:34.487945 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d3ef7-52ed-4838-a0b0-95ccafa7c617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:34.564733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-b67d3ef7-52ed-4838-a0b0-95ccafa7c617 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:34.565939 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b67d3ef7-52ed-4838-a0b0-95ccafa7c617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:34.597968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:34.666710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 555/1095] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:34.815977 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad008d5-9afa-47a9-8f62-52dad73148b3 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:34.817448 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4ad008d5-9afa-47a9-8f62-52dad73148b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:34.961621 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad008d5-9afa-47a9-8f62-52dad73148b3 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:35.001940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:35.074332 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a434fe6-0162-435d-b423-373fa38a1b0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:35.075001 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a434fe6-0162-435d-b423-373fa38a1b0e service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:35.075174 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a434fe6-0162-435d-b423-373fa38a1b0e service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:35.075443 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a434fe6-0162-435d-b423-373fa38a1b0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:35.077033 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5a434fe6-0162-435d-b423-373fa38a1b0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:35.168702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a434fe6-0162-435d-b423-373fa38a1b0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:35.170318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a434fe6-0162-435d-b423-373fa38a1b0e service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:35.188638 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 541/1096] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:36.324998 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e82fba-70ee-49f7-a599-109500e36b51 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:36.325543 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e82fba-70ee-49f7-a599-109500e36b51 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:36.325543 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e82fba-70ee-49f7-a599-109500e36b51 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:36.326953 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 556/1097] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:36.334203 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ecc5c-d506-4856-b16b-5431734ccb6f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:36.334666 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ecc5c-d506-4856-b16b-5431734ccb6f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:36.334769 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ecc5c-d506-4856-b16b-5431734ccb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:36.387241 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:36.409715 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8d2ecc5c-d506-4856-b16b-5431734ccb6f None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:36.410963 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8d2ecc5c-d506-4856-b16b-5431734ccb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:36.506680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 542/1098] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:36.657651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e82fba-70ee-49f7-a599-109500e36b51 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:36.659306 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-62e82fba-70ee-49f7-a599-109500e36b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:36.801587 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e82fba-70ee-49f7-a599-109500e36b51 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:36.813493 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:36.914071 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc60748-4c4f-4f9f-bb6a-6e8a1c7eb74a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:36.914607 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc60748-4c4f-4f9f-bb6a-6e8a1c7eb74a service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:36.914911 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc60748-4c4f-4f9f-bb6a-6e8a1c7eb74a service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:36.915188 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc60748-4c4f-4f9f-bb6a-6e8a1c7eb74a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:36.916760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-acc60748-4c4f-4f9f-bb6a-6e8a1c7eb74a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:37.003978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc60748-4c4f-4f9f-bb6a-6e8a1c7eb74a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:37.005549 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc60748-4c4f-4f9f-bb6a-6e8a1c7eb74a service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:37.023536 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 557/1099] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:38.144685 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303bdbe-f263-46ba-90a4-89135c7a4a15 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:38.145241 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303bdbe-f263-46ba-90a4-89135c7a4a15 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:38.145241 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303bdbe-f263-46ba-90a4-89135c7a4a15 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:38.146579 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 543/1100] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:38.152942 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a09d1e-9826-46cf-ae29-819d719648a0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:38.153383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a09d1e-9826-46cf-ae29-819d719648a0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:38.153549 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a09d1e-9826-46cf-ae29-819d719648a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:38.173841 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:38.230738 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-39a09d1e-9826-46cf-ae29-819d719648a0 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:38.231979 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-39a09d1e-9826-46cf-ae29-819d719648a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:38.327085 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 558/1101] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:38.479204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303bdbe-f263-46ba-90a4-89135c7a4a15 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:38.480832 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6303bdbe-f263-46ba-90a4-89135c7a4a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:38.591483 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:38.628046 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303bdbe-f263-46ba-90a4-89135c7a4a15 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:38.740290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97cd84-6c05-4f87-ac7b-e739fc599203 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:38.740896 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97cd84-6c05-4f87-ac7b-e739fc599203 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:38.741177 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97cd84-6c05-4f87-ac7b-e739fc599203 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:38.741451 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97cd84-6c05-4f87-ac7b-e739fc599203 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:38.743069 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ba97cd84-6c05-4f87-ac7b-e739fc599203 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:38.818304 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:38.831562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba97cd84-6c05-4f87-ac7b-e739fc599203 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:38.833152 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba97cd84-6c05-4f87-ac7b-e739fc599203 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:38.851222 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 544/1102] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:39.975798 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e20b5-a383-4acd-aea6-0803a7669448 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:39.976393 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e20b5-a383-4acd-aea6-0803a7669448 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:39.976393 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e20b5-a383-4acd-aea6-0803a7669448 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:39.977678 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 559/1103] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:39.983772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45f1ef-4d3d-4c9b-9d52-1b4b3ef60c84 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:39.984181 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45f1ef-4d3d-4c9b-9d52-1b4b3ef60c84 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:39.984319 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45f1ef-4d3d-4c9b-9d52-1b4b3ef60c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:40.058955 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-df45f1ef-4d3d-4c9b-9d52-1b4b3ef60c84 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:40.060146 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-df45f1ef-4d3d-4c9b-9d52-1b4b3ef60c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:40.156200 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 545/1104] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:40.306918 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e20b5-a383-4acd-aea6-0803a7669448 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:40.308448 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-182e20b5-a383-4acd-aea6-0803a7669448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:40.391786 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:40.452930 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e20b5-a383-4acd-aea6-0803a7669448 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:40.567352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ab940-a4b4-43cb-88dd-d5053c2bc61e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:40.567983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ab940-a4b4-43cb-88dd-d5053c2bc61e service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:40.568260 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ab940-a4b4-43cb-88dd-d5053c2bc61e service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:40.568531 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ab940-a4b4-43cb-88dd-d5053c2bc61e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:40.570094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2d6ab940-a4b4-43cb-88dd-d5053c2bc61e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:40.618566 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:40.661711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ab940-a4b4-43cb-88dd-d5053c2bc61e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:40.663339 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ab940-a4b4-43cb-88dd-d5053c2bc61e service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:40.681269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 560/1105] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:41.728942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d6994-7080-4511-924b-6af814bd4dcf None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:41.729520 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d6994-7080-4511-924b-6af814bd4dcf None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:41.729520 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d6994-7080-4511-924b-6af814bd4dcf None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:41.730840 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 546/1106] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:41.737578 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3364fa-ab5b-4a3c-a32d-c3d3f330a056 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:41.738029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3364fa-ab5b-4a3c-a32d-c3d3f330a056 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:41.738132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3364fa-ab5b-4a3c-a32d-c3d3f330a056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:41.813666 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6c3364fa-ab5b-4a3c-a32d-c3d3f330a056 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:41.814831 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6c3364fa-ab5b-4a3c-a32d-c3d3f330a056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:41.909729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 561/1107] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:05:42.061426 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d6994-7080-4511-924b-6af814bd4dcf None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:42.062957 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-106d6994-7080-4511-924b-6af814bd4dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:42.113256 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:42.208937 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d6994-7080-4511-924b-6af814bd4dcf None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:42.324117 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f7c26-4da7-404c-a884-0984732451fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:42.324724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f7c26-4da7-404c-a884-0984732451fc service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:42.324998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f7c26-4da7-404c-a884-0984732451fc service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:42.325263 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f7c26-4da7-404c-a884-0984732451fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:42.326834 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b01f7c26-4da7-404c-a884-0984732451fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:42.334243 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:42.415533 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01f7c26-4da7-404c-a884-0984732451fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:42.417116 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01f7c26-4da7-404c-a884-0984732451fc service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:42.434457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 547/1108] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:43.480902 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96048d9d-d8fb-48f3-be34-54e76cb777b6 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:43.481480 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96048d9d-d8fb-48f3-be34-54e76cb777b6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:43.481480 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96048d9d-d8fb-48f3-be34-54e76cb777b6 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:43.482876 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 562/1109] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:43.489649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21981cdd-d254-4644-9909-6f7fcad13bb3 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:43.490118 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21981cdd-d254-4644-9909-6f7fcad13bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:43.490223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21981cdd-d254-4644-9909-6f7fcad13bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:43.567510 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-21981cdd-d254-4644-9909-6f7fcad13bb3 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:43.568726 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-21981cdd-d254-4644-9909-6f7fcad13bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:43.635564 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:43.666544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 548/1110] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:43.815767 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96048d9d-d8fb-48f3-be34-54e76cb777b6 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:43.817320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-96048d9d-d8fb-48f3-be34-54e76cb777b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:43.834573 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:43.962032 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96048d9d-d8fb-48f3-be34-54e76cb777b6 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:44.050067 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:44.075541 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6493b-5aa3-43e3-9d23-2f31ad624138 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:44.076147 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6493b-5aa3-43e3-9d23-2f31ad624138 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:44.076423 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6493b-5aa3-43e3-9d23-2f31ad624138 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:44.076688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6493b-5aa3-43e3-9d23-2f31ad624138 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:44.078247 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-32f6493b-5aa3-43e3-9d23-2f31ad624138 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:44.174533 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6493b-5aa3-43e3-9d23-2f31ad624138 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:44.176094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6493b-5aa3-43e3-9d23-2f31ad624138 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:44.194119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 563/1111] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:45.269788 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04a615-1328-469b-870d-911d622a9997 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:45.270381 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04a615-1328-469b-870d-911d622a9997 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:45.270381 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04a615-1328-469b-870d-911d622a9997 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:45.271693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 549/1112] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:45.278488 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e8387-1ba9-4f08-9f05-356ab938f9ca None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:45.279074 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e8387-1ba9-4f08-9f05-356ab938f9ca None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:45.279074 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e8387-1ba9-4f08-9f05-356ab938f9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:45.353492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d83e8387-1ba9-4f08-9f05-356ab938f9ca None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:45.354687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d83e8387-1ba9-4f08-9f05-356ab938f9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:45.395660 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:45.450269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 564/1113] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:45.603541 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04a615-1328-469b-870d-911d622a9997 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:45.605037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7b04a615-1328-469b-870d-911d622a9997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:45.748421 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04a615-1328-469b-870d-911d622a9997 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:45.802260 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:45.859567 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96366158-2c58-4f10-9f34-c82ae935a3ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:45.860128 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96366158-2c58-4f10-9f34-c82ae935a3ad service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:45.860463 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96366158-2c58-4f10-9f34-c82ae935a3ad service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:45.860673 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96366158-2c58-4f10-9f34-c82ae935a3ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:45.862233 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-96366158-2c58-4f10-9f34-c82ae935a3ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:45.950573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96366158-2c58-4f10-9f34-c82ae935a3ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:45.952196 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96366158-2c58-4f10-9f34-c82ae935a3ad service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:45.970799 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 550/1114] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:47.020836 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218509d9-d8af-4c23-a2a0-4f8e3b0ad674 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:47.021401 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218509d9-d8af-4c23-a2a0-4f8e3b0ad674 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:47.021401 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218509d9-d8af-4c23-a2a0-4f8e3b0ad674 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:47.022718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 565/1115] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:47.028911 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161584d-f391-4f17-bd63-428fbe1c5dd0 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:47.029316 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161584d-f391-4f17-bd63-428fbe1c5dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:47.029449 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161584d-f391-4f17-bd63-428fbe1c5dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:47.104898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9161584d-f391-4f17-bd63-428fbe1c5dd0 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:47.106111 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9161584d-f391-4f17-bd63-428fbe1c5dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:47.108137 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:47.209385 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 551/1116] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:05:47.360998 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218509d9-d8af-4c23-a2a0-4f8e3b0ad674 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:47.362509 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-218509d9-d8af-4c23-a2a0-4f8e3b0ad674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:47.506107 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218509d9-d8af-4c23-a2a0-4f8e3b0ad674 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:47.533858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:47.618518 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6314d5c-1301-44a8-a488-7ee5ffb32c2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:47.619155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6314d5c-1301-44a8-a488-7ee5ffb32c2d service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:47.619439 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6314d5c-1301-44a8-a488-7ee5ffb32c2d service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:47.619711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6314d5c-1301-44a8-a488-7ee5ffb32c2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:47.621296 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d6314d5c-1301-44a8-a488-7ee5ffb32c2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:47.710972 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6314d5c-1301-44a8-a488-7ee5ffb32c2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:47.712587 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6314d5c-1301-44a8-a488-7ee5ffb32c2d service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:47.730467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 566/1117] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:49.785603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d834a-b362-4966-839c-5cd38f82529e None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:49.786192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d834a-b362-4966-839c-5cd38f82529e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:49.786192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d834a-b362-4966-839c-5cd38f82529e None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:49.787533 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 552/1118] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:49.792703 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1eae02-8cac-4066-9dab-d561e033306c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:49.793129 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1eae02-8cac-4066-9dab-d561e033306c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:49.793266 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1eae02-8cac-4066-9dab-d561e033306c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:49.834512 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:49.859787 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3b1eae02-8cac-4066-9dab-d561e033306c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:49.861754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3b1eae02-8cac-4066-9dab-d561e033306c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:49.957627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 567/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:50.242976 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d834a-b362-4966-839c-5cd38f82529e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:50.244540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0c7d834a-b362-4966-839c-5cd38f82529e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:50.349099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d834a-b362-4966-839c-5cd38f82529e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:50.380320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:50.421572 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eca2c6-b74c-48da-a05c-2ed88059a77e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:50.422171 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eca2c6-b74c-48da-a05c-2ed88059a77e service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:50.422470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eca2c6-b74c-48da-a05c-2ed88059a77e service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:50.423070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eca2c6-b74c-48da-a05c-2ed88059a77e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:50.424448 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d0eca2c6-b74c-48da-a05c-2ed88059a77e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:50.532558 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eca2c6-b74c-48da-a05c-2ed88059a77e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:50.533929 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eca2c6-b74c-48da-a05c-2ed88059a77e service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:50.551517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 553/1120] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:52.831847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0a59f-b971-484e-a599-d6584e6c18e1 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:52.832411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0a59f-b971-484e-a599-d6584e6c18e1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:52.832411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0a59f-b971-484e-a599-d6584e6c18e1 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:52.833787 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 568/1121] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:52.838989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354864b8-ba32-4c6d-b7c7-d60fa53481ff None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:52.839463 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354864b8-ba32-4c6d-b7c7-d60fa53481ff None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:52.839560 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354864b8-ba32-4c6d-b7c7-d60fa53481ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:52.905238 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-354864b8-ba32-4c6d-b7c7-d60fa53481ff None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:52.907627 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-354864b8-ba32-4c6d-b7c7-d60fa53481ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:52.920797 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:53.006496 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 554/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:53.293840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0a59f-b971-484e-a599-d6584e6c18e1 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:53.295426 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e4b0a59f-b971-484e-a599-d6584e6c18e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:53.397332 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0a59f-b971-484e-a599-d6584e6c18e1 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:53.410693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:53.471440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79101da3-40d0-407d-9d03-d88458c6107d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:53.472050 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79101da3-40d0-407d-9d03-d88458c6107d service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:53.472336 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79101da3-40d0-407d-9d03-d88458c6107d service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:53.472614 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79101da3-40d0-407d-9d03-d88458c6107d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:53.474213 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-79101da3-40d0-407d-9d03-d88458c6107d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:53.582463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79101da3-40d0-407d-9d03-d88458c6107d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:53.583938 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79101da3-40d0-407d-9d03-d88458c6107d service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:53.602160 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 569/1123] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:56.044696 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc44165-2e3c-4a9b-ba93-fc38c47486a9 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:56.045200 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc44165-2e3c-4a9b-ba93-fc38c47486a9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:56.045303 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc44165-2e3c-4a9b-ba93-fc38c47486a9 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:56.046564 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 555/1124] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:56.052578 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667449a-ac6e-4aff-90b0-29908ca14363 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:56.052999 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667449a-ac6e-4aff-90b0-29908ca14363 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:56.053154 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667449a-ac6e-4aff-90b0-29908ca14363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:56.100183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:56.118593 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a667449a-ac6e-4aff-90b0-29908ca14363 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:56.120077 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a667449a-ac6e-4aff-90b0-29908ca14363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:56.213468 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 570/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:56.497091 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc44165-2e3c-4a9b-ba93-fc38c47486a9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:56.498625 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-efc44165-2e3c-4a9b-ba93-fc38c47486a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:56.601353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc44165-2e3c-4a9b-ba93-fc38c47486a9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:56.640660 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:56.674823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e893bd2-ef24-4676-a515-90aae43d544e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:56.675450 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e893bd2-ef24-4676-a515-90aae43d544e service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:56.675724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e893bd2-ef24-4676-a515-90aae43d544e service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:56.675993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e893bd2-ef24-4676-a515-90aae43d544e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:56.677564 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4e893bd2-ef24-4676-a515-90aae43d544e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:56.784676 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e893bd2-ef24-4676-a515-90aae43d544e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:56.786125 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e893bd2-ef24-4676-a515-90aae43d544e service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:56.804175 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 556/1126] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:05:59.064897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e74c37-ab75-402f-9120-ad3459d76ee5 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:59.065436 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e74c37-ab75-402f-9120-ad3459d76ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:59.065436 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e74c37-ab75-402f-9120-ad3459d76ee5 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:59.066784 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 571/1127] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:05:59.072024 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efeac54-8b18-45fd-ab36-9339ae557602 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:59.072468 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efeac54-8b18-45fd-ab36-9339ae557602 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:59.072564 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efeac54-8b18-45fd-ab36-9339ae557602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:59.138457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-6efeac54-8b18-45fd-ab36-9339ae557602 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:05:59.140352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6efeac54-8b18-45fd-ab36-9339ae557602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:59.162390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:59.239485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 557/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:05:59.526295 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e74c37-ab75-402f-9120-ad3459d76ee5 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:05:59.527875 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d9e74c37-ab75-402f-9120-ad3459d76ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:59.631818 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e74c37-ab75-402f-9120-ad3459d76ee5 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:05:59.652970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:05:59.706430 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915a891-0abe-4475-ab2a-66cab708e10f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:05:59.707068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915a891-0abe-4475-ab2a-66cab708e10f service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:05:59.707355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915a891-0abe-4475-ab2a-66cab708e10f service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:05:59.707632 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915a891-0abe-4475-ab2a-66cab708e10f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:05:59.709230 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3915a891-0abe-4475-ab2a-66cab708e10f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:05:59.817296 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915a891-0abe-4475-ab2a-66cab708e10f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:05:59.818718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915a891-0abe-4475-ab2a-66cab708e10f service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:05:59.836308 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 572/1129] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:05:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:06:02.071136 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc5059-cd59-4ec0-943b-e39263e43aac None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:02.071681 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc5059-cd59-4ec0-943b-e39263e43aac None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:02.071681 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc5059-cd59-4ec0-943b-e39263e43aac None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:02.073027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 558/1130] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:06:02.078357 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0d375-9b0b-4a43-9109-008af0b0e18b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:02.079432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0d375-9b0b-4a43-9109-008af0b0e18b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:02.079613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0d375-9b0b-4a43-9109-008af0b0e18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:02.142254 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:02.145238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-58d0d375-9b0b-4a43-9109-008af0b0e18b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:06:02.147049 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-58d0d375-9b0b-4a43-9109-008af0b0e18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:02.242638 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 573/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:06:02.531274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc5059-cd59-4ec0-943b-e39263e43aac None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:06:02.533133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8bcc5059-cd59-4ec0-943b-e39263e43aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:02.636788 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc5059-cd59-4ec0-943b-e39263e43aac None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:06:02.685634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:02.711106 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3b6b2-bad6-4fb6-bc69-617c1ea43706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:06:02.711703 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3b6b2-bad6-4fb6-bc69-617c1ea43706 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:02.711981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3b6b2-bad6-4fb6-bc69-617c1ea43706 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:02.712271 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3b6b2-bad6-4fb6-bc69-617c1ea43706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:02.713835 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-42b3b6b2-bad6-4fb6-bc69-617c1ea43706 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:02.821682 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3b6b2-bad6-4fb6-bc69-617c1ea43706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:06:02.823149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3b6b2-bad6-4fb6-bc69-617c1ea43706 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:06:02.841537 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 559/1132] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:06:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:06:05.038268 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5b8f1-51ae-46ef-81b9-729851e53d97 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:05.038878 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5b8f1-51ae-46ef-81b9-729851e53d97 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:05.038878 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5b8f1-51ae-46ef-81b9-729851e53d97 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:05.040200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 574/1133] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:06:05.045404 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fa884-3251-4945-b906-f03daa0ffa0e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:05.045804 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fa884-3251-4945-b906-f03daa0ffa0e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:05.045963 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fa884-3251-4945-b906-f03daa0ffa0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:05.112259 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1f4fa884-3251-4945-b906-f03daa0ffa0e None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:06:05.113470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1f4fa884-3251-4945-b906-f03daa0ffa0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:05.142656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:05.210514 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 560/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:06:05.497372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5b8f1-51ae-46ef-81b9-729851e53d97 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:06:05.498920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b7c5b8f1-51ae-46ef-81b9-729851e53d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:05.602494 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5b8f1-51ae-46ef-81b9-729851e53d97 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:06:05.630292 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:05.675711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d4f168-9aa8-4770-a0b9-f1bb5dad92b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:06:05.676305 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4f168-9aa8-4770-a0b9-f1bb5dad92b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:05.676579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4f168-9aa8-4770-a0b9-f1bb5dad92b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:05.676847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4f168-9aa8-4770-a0b9-f1bb5dad92b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:05.678403 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-78d4f168-9aa8-4770-a0b9-f1bb5dad92b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:05.783433 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d4f168-9aa8-4770-a0b9-f1bb5dad92b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:06:05.784780 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d4f168-9aa8-4770-a0b9-f1bb5dad92b9 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:06:05.802570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 575/1135] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:06:07.993318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f273b-bbbe-4f50-ac5e-059a5796b064 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:07.993948 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f273b-bbbe-4f50-ac5e-059a5796b064 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:07.993948 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f273b-bbbe-4f50-ac5e-059a5796b064 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:07.995249 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 561/1136] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:06:08.000509 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76218156-06c7-4260-b1ca-01b9aba01533 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:08.000958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76218156-06c7-4260-b1ca-01b9aba01533 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:08.001197 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76218156-06c7-4260-b1ca-01b9aba01533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:08.068152 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-76218156-06c7-4260-b1ca-01b9aba01533 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:06:08.069360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-76218156-06c7-4260-b1ca-01b9aba01533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:08.077457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:08.164452 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 576/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:06:08.451289 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f273b-bbbe-4f50-ac5e-059a5796b064 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:06:08.452858 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ae0f273b-bbbe-4f50-ac5e-059a5796b064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:08.556306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f273b-bbbe-4f50-ac5e-059a5796b064 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:06:08.613496 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:08.631404 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5521b-eeaf-40b2-8713-7fd009af1014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:06:08.632034 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5521b-eeaf-40b2-8713-7fd009af1014 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:08.632313 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5521b-eeaf-40b2-8713-7fd009af1014 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:08.632582 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5521b-eeaf-40b2-8713-7fd009af1014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:08.634158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3aa5521b-eeaf-40b2-8713-7fd009af1014 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:08.741816 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5521b-eeaf-40b2-8713-7fd009af1014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:06:08.743341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5521b-eeaf-40b2-8713-7fd009af1014 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:06:08.761729 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 562/1138] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:06:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:06:10.964325 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be92996-ef53-40bb-acec-c0330be69546 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:10.964861 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be92996-ef53-40bb-acec-c0330be69546 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:10.964861 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be92996-ef53-40bb-acec-c0330be69546 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:10.966239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 577/1139] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:06:10.972243 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce27e0-177d-4af3-b97e-41b47e957431 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:10.972670 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce27e0-177d-4af3-b97e-41b47e957431 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:10.972796 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce27e0-177d-4af3-b97e-41b47e957431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:11.038006 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-61ce27e0-177d-4af3-b97e-41b47e957431 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:06:11.039238 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-61ce27e0-177d-4af3-b97e-41b47e957431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:11.077244 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:11.134426 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 563/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:06:11.421833 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be92996-ef53-40bb-acec-c0330be69546 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:06:11.423565 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8be92996-ef53-40bb-acec-c0330be69546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:11.525745 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be92996-ef53-40bb-acec-c0330be69546 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:06:11.564293 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:11.600933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a952c-c294-46ee-bd23-750366aad1d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:06:11.601537 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a952c-c294-46ee-bd23-750366aad1d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:11.601816 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a952c-c294-46ee-bd23-750366aad1d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:11.602091 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a952c-c294-46ee-bd23-750366aad1d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:11.603728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fa7a952c-c294-46ee-bd23-750366aad1d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:11.711992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a952c-c294-46ee-bd23-750366aad1d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:06:11.713415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a952c-c294-46ee-bd23-750366aad1d2 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:06:11.731581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 578/1141] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:06:13.885916 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3f3b4-ec8a-4489-9b54-4b382119866f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:13.886507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3f3b4-ec8a-4489-9b54-4b382119866f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:13.886507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3f3b4-ec8a-4489-9b54-4b382119866f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:13.887833 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 564/1142] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:06:13.893441 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf2786-8ca8-46c5-95e4-ecc9b7a61f66 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:13.893845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf2786-8ca8-46c5-95e4-ecc9b7a61f66 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:13.894129 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf2786-8ca8-46c5-95e4-ecc9b7a61f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:13.961309 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-dddf2786-8ca8-46c5-95e4-ecc9b7a61f66 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:06:13.962565 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dddf2786-8ca8-46c5-95e4-ecc9b7a61f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:13.980956 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:14.059495 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 579/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:06:14.345290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3f3b4-ec8a-4489-9b54-4b382119866f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:06:14.346859 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e0f3f3b4-ec8a-4489-9b54-4b382119866f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:14.449135 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3f3b4-ec8a-4489-9b54-4b382119866f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:06:14.516893 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:14.523824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d45b6a-5d60-4849-84dd-027c720a6230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:06:14.524429 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d45b6a-5d60-4849-84dd-027c720a6230 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:14.524704 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d45b6a-5d60-4849-84dd-027c720a6230 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:14.524970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d45b6a-5d60-4849-84dd-027c720a6230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:14.526563 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e6d45b6a-5d60-4849-84dd-027c720a6230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:14.632621 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d45b6a-5d60-4849-84dd-027c720a6230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:06:14.634102 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d45b6a-5d60-4849-84dd-027c720a6230 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:06:14.652766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 565/1144] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:06:16.899564 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cfb50-ddd4-4944-9d43-632ddb82c566 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:16.900131 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cfb50-ddd4-4944-9d43-632ddb82c566 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:16.900131 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cfb50-ddd4-4944-9d43-632ddb82c566 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:16.901434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 580/1145] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:06:16.907454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1264f-f37e-4a57-a64c-17dc3422180c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:16.907913 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1264f-f37e-4a57-a64c-17dc3422180c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:16.908249 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1264f-f37e-4a57-a64c-17dc3422180c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:16.974127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-70e1264f-f37e-4a57-a64c-17dc3422180c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:06:16.975365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-70e1264f-f37e-4a57-a64c-17dc3422180c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:17.023569 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:17.072829 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 566/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:06:17.358572 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cfb50-ddd4-4944-9d43-632ddb82c566 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:06:17.360389 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-642cfb50-ddd4-4944-9d43-632ddb82c566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:17.462347 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cfb50-ddd4-4944-9d43-632ddb82c566 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:06:17.509874 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:17.536440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b622553-03ff-4914-8772-0da7541470ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:06:17.537005 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b622553-03ff-4914-8772-0da7541470ee service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:17.537264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b622553-03ff-4914-8772-0da7541470ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:17.537539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b622553-03ff-4914-8772-0da7541470ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:17.539162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5b622553-03ff-4914-8772-0da7541470ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:17.647339 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b622553-03ff-4914-8772-0da7541470ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:06:17.648766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b622553-03ff-4914-8772-0da7541470ee service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:06:17.666326 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 581/1147] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:06:27.071324 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=22 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:27.664107 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=66 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:37.648834 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad8091-12eb-49a5-abd9-63cadcd374c3 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:37.649501 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad8091-12eb-49a5-abd9-63cadcd374c3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:37.649501 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad8091-12eb-49a5-abd9-63cadcd374c3 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:37.650807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 567/1148] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:06:37.656062 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d665346-1067-49ef-9005-2bc3f58b8c73 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:37.657102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d665346-1067-49ef-9005-2bc3f58b8c73 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:37.657216 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d665346-1067-49ef-9005-2bc3f58b8c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:37.721572 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8d665346-1067-49ef-9005-2bc3f58b8c73 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:06:37.722760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8d665346-1067-49ef-9005-2bc3f58b8c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:37.816509 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 582/1149] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:06:38.110747 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e37ea-1eb8-44cc-8fb0-89c7350e2e75 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:38.111063 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e37ea-1eb8-44cc-8fb0-89c7350e2e75 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:38.111337 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e37ea-1eb8-44cc-8fb0-89c7350e2e75 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:38.112663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 568/1150] 10.222.0.27 () {54 vars in 962 bytes} [Mon Oct 17 03:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:06:38.117608 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c20cc-eebe-4280-9fd4-592fb17c94fb None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:38.117897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c20cc-eebe-4280-9fd4-592fb17c94fb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:38.118172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c20cc-eebe-4280-9fd4-592fb17c94fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:38.191654 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-fc0c20cc-eebe-4280-9fd4-592fb17c94fb None None] MFA Rules not processed for user `a8aa29639a3143d2a270e3248d2205d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:06:38.192830 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fc0c20cc-eebe-4280-9fd4-592fb17c94fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:38.213024 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:38.268618 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 583/1151] 10.222.0.27 () {58 vars in 1059 bytes} [Mon Oct 17 03:06:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:06:38.276395 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e37ea-1eb8-44cc-8fb0-89c7350e2e75 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:06:38.277939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-699e37ea-1eb8-44cc-8fb0-89c7350e2e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:38.381758 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e37ea-1eb8-44cc-8fb0-89c7350e2e75 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:06:38.455162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf0ac8-f85f-4e14-a3d5-625a758538e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8aa29639a3143d2a270e3248d2205d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:06:38.455766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf0ac8-f85f-4e14-a3d5-625a758538e1 service glance] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:38.456044 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf0ac8-f85f-4e14-a3d5-625a758538e1 service glance] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:38.456311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf0ac8-f85f-4e14-a3d5-625a758538e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:38.457877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-29cf0ac8-f85f-4e14-a3d5-625a758538e1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:38.513291 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:38.568429 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf0ac8-f85f-4e14-a3d5-625a758538e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:06:38.569863 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf0ac8-f85f-4e14-a3d5-625a758538e1 service glance] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:06:38.587732 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 569/1152] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:06:40.824583 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d00a3-c234-47a9-acf7-2226bc0186fe None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:40.825218 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d00a3-c234-47a9-acf7-2226bc0186fe None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:40.825218 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d00a3-c234-47a9-acf7-2226bc0186fe None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:40.826539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 584/1153] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:06:40.832048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46b1d9-f8fe-4352-ab00-c106f7ec9ac0 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:40.832447 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46b1d9-f8fe-4352-ab00-c106f7ec9ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:40.832599 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46b1d9-f8fe-4352-ab00-c106f7ec9ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:40.897612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4a46b1d9-f8fe-4352-ab00-c106f7ec9ac0 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:06:40.898821 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4a46b1d9-f8fe-4352-ab00-c106f7ec9ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:40.998043 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 570/1154] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:06:41.270790 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d00a3-c234-47a9-acf7-2226bc0186fe None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:06:41.272259 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cb7d00a3-c234-47a9-acf7-2226bc0186fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:41.395482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d00a3-c234-47a9-acf7-2226bc0186fe None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:06:41.448119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:41.495766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a74b1-9184-4ce3-a1e3-3c0ba45a1397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:06:41.496388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a74b1-9184-4ce3-a1e3-3c0ba45a1397 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:41.496671 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a74b1-9184-4ce3-a1e3-3c0ba45a1397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:41.496943 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a74b1-9184-4ce3-a1e3-3c0ba45a1397 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:41.497580 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a74b1-9184-4ce3-a1e3-3c0ba45a1397 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:06:41.498685 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-327a74b1-9184-4ce3-a1e3-3c0ba45a1397 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:06:41.498685 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Oct 17 03:06:41.498685 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: Oct 17 03:06:41.499861 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a74b1-9184-4ce3-a1e3-3c0ba45a1397 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:06:41.519839 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 585/1155] 10.222.0.27 () {56 vars in 1108 bytes} [Mon Oct 17 03:06:41 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:06:43.680334 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69c3b9-1377-40ca-95d6-51a31013be21 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:43.680901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69c3b9-1377-40ca-95d6-51a31013be21 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:43.680901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69c3b9-1377-40ca-95d6-51a31013be21 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:43.682210 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 571/1156] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:06:43.687462 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc5f60-fc38-4369-ba6a-1d068dce2fb9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:43.688288 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc5f60-fc38-4369-ba6a-1d068dce2fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:43.688288 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc5f60-fc38-4369-ba6a-1d068dce2fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:43.753888 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-41bc5f60-fc38-4369-ba6a-1d068dce2fb9 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:06:43.755134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-41bc5f60-fc38-4369-ba6a-1d068dce2fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:43.852937 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 586/1157] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:06:44.141860 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69c3b9-1377-40ca-95d6-51a31013be21 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:06:44.143650 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6f69c3b9-1377-40ca-95d6-51a31013be21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:44.153158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:44.288731 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69c3b9-1377-40ca-95d6-51a31013be21 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:06:44.367485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:44.401224 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde6a3c-6289-4e15-9bab-d17ec011d7f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:06:44.401820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde6a3c-6289-4e15-9bab-d17ec011d7f3 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:44.402095 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde6a3c-6289-4e15-9bab-d17ec011d7f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:44.402362 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde6a3c-6289-4e15-9bab-d17ec011d7f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:44.403968 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3cde6a3c-6289-4e15-9bab-d17ec011d7f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:44.492795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde6a3c-6289-4e15-9bab-d17ec011d7f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:06:44.494401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde6a3c-6289-4e15-9bab-d17ec011d7f3 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:06:44.512075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 572/1158] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:06:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:06:46.821745 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85221cb6-831d-4609-b5b1-3567164a33f8 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:46.822333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85221cb6-831d-4609-b5b1-3567164a33f8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:46.822333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85221cb6-831d-4609-b5b1-3567164a33f8 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:46.823642 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 587/1159] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:06:46.828912 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b7034-30da-4270-8c9f-133f74e8476d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:46.829347 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b7034-30da-4270-8c9f-133f74e8476d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:46.829454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b7034-30da-4270-8c9f-133f74e8476d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:46.894399 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-497b7034-30da-4270-8c9f-133f74e8476d None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:06:46.895624 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-497b7034-30da-4270-8c9f-133f74e8476d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:46.941306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:46.990643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 573/1160] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:06:47.279903 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85221cb6-831d-4609-b5b1-3567164a33f8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:06:47.281561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-85221cb6-831d-4609-b5b1-3567164a33f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:47.305438 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:47.384108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85221cb6-831d-4609-b5b1-3567164a33f8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:06:47.457387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103650b3-ee15-483a-b9e9-2dbfe46024a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:06:47.458056 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103650b3-ee15-483a-b9e9-2dbfe46024a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:47.458449 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103650b3-ee15-483a-b9e9-2dbfe46024a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:47.458978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103650b3-ee15-483a-b9e9-2dbfe46024a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:47.461845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-103650b3-ee15-483a-b9e9-2dbfe46024a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:47.537493 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:47.572191 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103650b3-ee15-483a-b9e9-2dbfe46024a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:06:47.573614 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103650b3-ee15-483a-b9e9-2dbfe46024a1 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:06:47.591418 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 588/1161] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:06:49.820630 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ffb5f-a59c-40e1-a7d7-e3e07626c0f9 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:49.821200 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ffb5f-a59c-40e1-a7d7-e3e07626c0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:49.821200 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ffb5f-a59c-40e1-a7d7-e3e07626c0f9 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:49.822520 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 574/1162] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:06:49.827861 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41afa16-c72d-49e5-8d37-65138d53b130 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:49.828338 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41afa16-c72d-49e5-8d37-65138d53b130 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:49.828451 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41afa16-c72d-49e5-8d37-65138d53b130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:49.894446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e41afa16-c72d-49e5-8d37-65138d53b130 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:06:49.895680 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e41afa16-c72d-49e5-8d37-65138d53b130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:49.991703 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 589/1163] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:06:50.284742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ffb5f-a59c-40e1-a7d7-e3e07626c0f9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:06:50.286339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-751ffb5f-a59c-40e1-a7d7-e3e07626c0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:50.388704 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ffb5f-a59c-40e1-a7d7-e3e07626c0f9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:06:50.458522 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:50.459992 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9477e-6469-4808-8a19-c2c414228058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:06:50.460587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9477e-6469-4808-8a19-c2c414228058 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:50.461044 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9477e-6469-4808-8a19-c2c414228058 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:50.461523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9477e-6469-4808-8a19-c2c414228058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:50.463425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-38a9477e-6469-4808-8a19-c2c414228058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:50.570832 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9477e-6469-4808-8a19-c2c414228058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:06:50.572290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9477e-6469-4808-8a19-c2c414228058 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:06:50.589693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 575/1164] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:06:52.906368 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ef587-ff51-43be-98e9-f7c2f91ff54f None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:52.906983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ef587-ff51-43be-98e9-f7c2f91ff54f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:52.906983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ef587-ff51-43be-98e9-f7c2f91ff54f None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:52.908795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 590/1165] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:06:52.915072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f1756-68ac-4ac3-9f72-cd7393854b27 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:52.915491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f1756-68ac-4ac3-9f72-cd7393854b27 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:52.915615 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f1756-68ac-4ac3-9f72-cd7393854b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:52.979740 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8c1f1756-68ac-4ac3-9f72-cd7393854b27 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:06:52.980930 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8c1f1756-68ac-4ac3-9f72-cd7393854b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:53.031905 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:53.076655 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 576/1166] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:06:53.366329 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ef587-ff51-43be-98e9-f7c2f91ff54f None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:06:53.367915 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5b4ef587-ff51-43be-98e9-f7c2f91ff54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:53.404530 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:53.473616 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ef587-ff51-43be-98e9-f7c2f91ff54f None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:06:53.547530 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7d7ae-6aa3-47b1-b023-83e861f99649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:06:53.548145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7d7ae-6aa3-47b1-b023-83e861f99649 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:53.548428 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7d7ae-6aa3-47b1-b023-83e861f99649 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:53.548779 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7d7ae-6aa3-47b1-b023-83e861f99649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:53.550375 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-81b7d7ae-6aa3-47b1-b023-83e861f99649 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:53.630646 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:53.661502 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7d7ae-6aa3-47b1-b023-83e861f99649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:06:53.662969 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7d7ae-6aa3-47b1-b023-83e861f99649 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:06:53.681542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 591/1167] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:06:56.012264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f885bdc-d0dd-4788-818e-29165f8d7852 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:56.012844 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f885bdc-d0dd-4788-818e-29165f8d7852 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:56.012844 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f885bdc-d0dd-4788-818e-29165f8d7852 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:56.014142 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 577/1168] 10.222.0.27 () {54 vars in 927 bytes} [Mon Oct 17 03:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:06:56.019674 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f575fa0-c39d-43cd-b2c7-6de4e7aabd16 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:56.020093 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f575fa0-c39d-43cd-b2c7-6de4e7aabd16 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:56.020242 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f575fa0-c39d-43cd-b2c7-6de4e7aabd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:56.087394 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3f575fa0-c39d-43cd-b2c7-6de4e7aabd16 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:06:56.088606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3f575fa0-c39d-43cd-b2c7-6de4e7aabd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:56.184011 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 592/1169] 10.222.0.27 () {58 vars in 1024 bytes} [Mon Oct 17 03:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:06:56.473558 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f885bdc-d0dd-4788-818e-29165f8d7852 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:06:56.475129 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7f885bdc-d0dd-4788-818e-29165f8d7852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:56.576241 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f885bdc-d0dd-4788-818e-29165f8d7852 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:06:56.649804 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c4cb3-1987-42fe-a821-8eec618772db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:06:56.650405 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c4cb3-1987-42fe-a821-8eec618772db service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:06:56.650688 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c4cb3-1987-42fe-a821-8eec618772db service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:06:56.650985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c4cb3-1987-42fe-a821-8eec618772db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:06:56.652575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-454c4cb3-1987-42fe-a821-8eec618772db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:06:56.661460 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:06:56.761947 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c4cb3-1987-42fe-a821-8eec618772db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:06:56.763438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c4cb3-1987-42fe-a821-8eec618772db service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:06:56.781256 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 578/1170] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:07:06.182486 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=82 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:07:06.779205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=50 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:08:09.838401 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd3813-12c5-4f7d-803d-6f81b124b2d4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:08:09.839047 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd3813-12c5-4f7d-803d-6f81b124b2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:08:09.839047 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd3813-12c5-4f7d-803d-6f81b124b2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:08:09.918269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-10fd3813-12c5-4f7d-803d-6f81b124b2d4 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:08:09.919547 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-10fd3813-12c5-4f7d-803d-6f81b124b2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:08:10.015413 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 593/1171] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:08:10.024091 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610a5e6-257b-488e-b096-0f4a81b53063 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:08:10.024579 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610a5e6-257b-488e-b096-0f4a81b53063 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:08:10.024683 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610a5e6-257b-488e-b096-0f4a81b53063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:08:10.098839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-6610a5e6-257b-488e-b096-0f4a81b53063 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:08:10.099978 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6610a5e6-257b-488e-b096-0f4a81b53063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:08:10.195873 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 579/1172] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:08:10.200413 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd3813-12c5-4f7d-803d-6f81b124b2d4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:08:10.201598 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-10fd3813-12c5-4f7d-803d-6f81b124b2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:08:10.212721 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd3813-12c5-4f7d-803d-6f81b124b2d4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:08:10.272928 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:08:10.307762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221cab3-1d09-446a-b523-55b5babfeafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:08:10.308367 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221cab3-1d09-446a-b523-55b5babfeafd admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:08:10.308640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221cab3-1d09-446a-b523-55b5babfeafd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:08:10.308904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221cab3-1d09-446a-b523-55b5babfeafd admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:08:10.309537 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221cab3-1d09-446a-b523-55b5babfeafd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:08:10.309834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221cab3-1d09-446a-b523-55b5babfeafd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:08:10.310829 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-7221cab3-1d09-446a-b523-55b5babfeafd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:08:10.310829 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Oct 17 03:08:10.310829 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: Oct 17 03:08:10.311725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221cab3-1d09-446a-b523-55b5babfeafd admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:08:10.322688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 594/1173] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:08:10.332223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610a5e6-257b-488e-b096-0f4a81b53063 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:08:10.341269 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610a5e6-257b-488e-b096-0f4a81b53063 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:08:10.403283 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:08:10.436394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f447d-2367-42cc-9a17-9e435a6c8680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:08:10.436994 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f447d-2367-42cc-9a17-9e435a6c8680 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:08:10.437274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f447d-2367-42cc-9a17-9e435a6c8680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:08:10.437542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f447d-2367-42cc-9a17-9e435a6c8680 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:08:10.438172 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f447d-2367-42cc-9a17-9e435a6c8680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:08:10.439349 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-881f447d-2367-42cc-9a17-9e435a6c8680 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:08:10.439349 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Oct 17 03:08:10.439349 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: Oct 17 03:08:10.440548 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881f447d-2367-42cc-9a17-9e435a6c8680 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:08:10.495928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 580/1174] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:08:10 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:08:10.500041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221cab3-1d09-446a-b523-55b5babfeafd admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:08:10.501611 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7221cab3-1d09-446a-b523-55b5babfeafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:08:10.535920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221cab3-1d09-446a-b523-55b5babfeafd admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:08:10.628544 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3c83c-4b87-4cb4-8fc8-136607df9457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:08:10.629152 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3c83c-4b87-4cb4-8fc8-136607df9457 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:08:10.629421 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3c83c-4b87-4cb4-8fc8-136607df9457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:08:10.629684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3c83c-4b87-4cb4-8fc8-136607df9457 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:08:10.630308 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3c83c-4b87-4cb4-8fc8-136607df9457 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:08:10.631335 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-90c3c83c-4b87-4cb4-8fc8-136607df9457 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:08:10.631335 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Oct 17 03:08:10.631335 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: Oct 17 03:08:10.632632 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3c83c-4b87-4cb4-8fc8-136607df9457 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:08:10.678813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 595/1175] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:08:10 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:08:10.682853 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f447d-2367-42cc-9a17-9e435a6c8680 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:08:10.684474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-881f447d-2367-42cc-9a17-9e435a6c8680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:08:10.746186 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f447d-2367-42cc-9a17-9e435a6c8680 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:08:10.828672 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:08:10.840779 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3646c-b7f3-4968-bb5e-790ba0c02116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:08:10.841379 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3646c-b7f3-4968-bb5e-790ba0c02116 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:08:10.841654 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3646c-b7f3-4968-bb5e-790ba0c02116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:08:10.841920 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3646c-b7f3-4968-bb5e-790ba0c02116 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:08:10.842556 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda3646c-b7f3-4968-bb5e-790ba0c02116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:08:10.844173 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda3646c-b7f3-4968-bb5e-790ba0c02116 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:08:10.856299 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 581/1176] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:08:10.860581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3c83c-4b87-4cb4-8fc8-136607df9457 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:08:10.862077 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-90c3c83c-4b87-4cb4-8fc8-136607df9457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:08:10.873186 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3c83c-4b87-4cb4-8fc8-136607df9457 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:08:10.899049 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:08:10.971853 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e1970-2548-4392-b3db-16474cc415c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:08:10.972470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e1970-2548-4392-b3db-16474cc415c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:08:10.972750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e1970-2548-4392-b3db-16474cc415c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:08:10.973021 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e1970-2548-4392-b3db-16474cc415c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:08:10.973667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e1970-2548-4392-b3db-16474cc415c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:08:10.975283 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e1970-2548-4392-b3db-16474cc415c1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:08:10.988516 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 596/1177] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:08:10.992750 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3646c-b7f3-4968-bb5e-790ba0c02116 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:08:10.994286 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cda3646c-b7f3-4968-bb5e-790ba0c02116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:08:11.028177 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3646c-b7f3-4968-bb5e-790ba0c02116 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:08:11.122552 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dc1ec-0c06-4653-8ded-2c3165f86b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:08:11.123227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dc1ec-0c06-4653-8ded-2c3165f86b9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:08:11.123506 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dc1ec-0c06-4653-8ded-2c3165f86b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:08:11.123779 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dc1ec-0c06-4653-8ded-2c3165f86b9f admin admin] PATH_INFO: `/v3/projects/b277e196181a4f889070a47154446238/users/9fd1b268a9b94f91b0dc0ad09fcc1be7/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:08:11.147116 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512dc1ec-0c06-4653-8ded-2c3165f86b9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b277e196181a4f889070a47154446238, user_id=9fd1b268a9b94f91b0dc0ad09fcc1be7, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:08:11.149769 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512dc1ec-0c06-4653-8ded-2c3165f86b9f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:08:11.174026 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 582/1178] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:08:10 2022] PUT /identity/v3/projects/b277e196181a4f889070a47154446238/users/9fd1b268a9b94f91b0dc0ad09fcc1be7/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:08:11.185077 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5ac1b-4adc-4817-a33d-37a7217cc874 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:08:11.185365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5ac1b-4adc-4817-a33d-37a7217cc874 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:08:11.185642 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a5ac1b-4adc-4817-a33d-37a7217cc874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:08:11.245180 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f8a5ac1b-4adc-4817-a33d-37a7217cc874 None None] MFA Rules not processed for user `9fd1b268a9b94f91b0dc0ad09fcc1be7`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:08:11.246302 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f8a5ac1b-4adc-4817-a33d-37a7217cc874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:08:11.322453 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 597/1179] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:08:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:08:11.336552 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ded95-8b73-429d-9537-ce6157f751df None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:08:11.336839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ded95-8b73-429d-9537-ce6157f751df None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:08:11.337115 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ded95-8b73-429d-9537-ce6157f751df None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:08:11.338448 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 583/1180] 10.222.0.27 () {54 vars in 963 bytes} [Mon Oct 17 03:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:08:11.343290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27a6a1-476c-43f3-929b-26de96e09c22 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:08:11.343580 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27a6a1-476c-43f3-929b-26de96e09c22 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:08:11.343856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27a6a1-476c-43f3-929b-26de96e09c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:08:11.375205 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:08:11.419749 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-fb27a6a1-476c-43f3-929b-26de96e09c22 None None] MFA Rules not processed for user `67abf21b99d349dfa7d88a0e18be1fed`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:08:11.420940 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fb27a6a1-476c-43f3-929b-26de96e09c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:08:11.494527 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 598/1181] 10.222.0.27 () {58 vars in 1060 bytes} [Mon Oct 17 03:08:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:08:11.502913 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ded95-8b73-429d-9537-ce6157f751df None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:08:11.504385 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-385ded95-8b73-429d-9537-ce6157f751df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:08:11.554173 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:08:11.607314 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ded95-8b73-429d-9537-ce6157f751df None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:08:11.680652 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258e14b-5dca-4523-8d49-3e1ddf04e707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:08:11.681256 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258e14b-5dca-4523-8d49-3e1ddf04e707 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:08:11.681533 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258e14b-5dca-4523-8d49-3e1ddf04e707 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:08:11.681797 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258e14b-5dca-4523-8d49-3e1ddf04e707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:08:11.683400 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8258e14b-5dca-4523-8d49-3e1ddf04e707 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:08:11.779765 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258e14b-5dca-4523-8d49-3e1ddf04e707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:08:11.781130 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258e14b-5dca-4523-8d49-3e1ddf04e707 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:08:11.785141 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:08:11.800342 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 584/1182] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:08:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:08:11.845471 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b2d01-793f-4a44-8942-0c4aa49e8ff2 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:08:11.845750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b2d01-793f-4a44-8942-0c4aa49e8ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:08:11.846051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b2d01-793f-4a44-8942-0c4aa49e8ff2 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:08:11.847317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 599/1183] 10.222.0.27 () {52 vars in 830 bytes} [Mon Oct 17 03:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:08:11.850807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258e14b-5dca-4523-8d49-3e1ddf04e707 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:08:11.852064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8258e14b-5dca-4523-8d49-3e1ddf04e707 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:08:11.953780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258e14b-5dca-4523-8d49-3e1ddf04e707 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:08:12.046845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f28c30-e0c4-4dd2-8e0a-160d4a9d3b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:08:12.047492 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f28c30-e0c4-4dd2-8e0a-160d4a9d3b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:08:12.048891 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f28c30-e0c4-4dd2-8e0a-160d4a9d3b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:08:12.048891 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f28c30-e0c4-4dd2-8e0a-160d4a9d3b2a admin admin] PATH_INFO: `/v3/users/9fd1b268a9b94f91b0dc0ad09fcc1be7` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:08:12.061786 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f28c30-e0c4-4dd2-8e0a-160d4a9d3b2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fd1b268a9b94f91b0dc0ad09fcc1be7)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:08:12.062819 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-50f28c30-e0c4-4dd2-8e0a-160d4a9d3b2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:08:12.062819 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Oct 17 03:08:12.062819 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: Oct 17 03:08:12.064549 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f28c30-e0c4-4dd2-8e0a-160d4a9d3b2a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:08:12.074981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:08:12.132385 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-50f28c30-e0c4-4dd2-8e0a-160d4a9d3b2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fd1b268a9b94f91b0dc0ad09fcc1be7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:08:12.148086 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-50f28c30-e0c4-4dd2-8e0a-160d4a9d3b2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fd1b268a9b94f91b0dc0ad09fcc1be7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:08:12.148535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-50f28c30-e0c4-4dd2-8e0a-160d4a9d3b2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fd1b268a9b94f91b0dc0ad09fcc1be7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:08:12.158131 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-50f28c30-e0c4-4dd2-8e0a-160d4a9d3b2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fd1b268a9b94f91b0dc0ad09fcc1be7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:08:12.181234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-50f28c30-e0c4-4dd2-8e0a-160d4a9d3b2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fd1b268a9b94f91b0dc0ad09fcc1be7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:08:12.187106 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 585/1184] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:08:11 2022] DELETE /identity/v3/users/9fd1b268a9b94f91b0dc0ad09fcc1be7 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:08:12.192228 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621e18b-1fbc-4d0f-bd15-3cb30ba20e67 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:08:12.192698 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621e18b-1fbc-4d0f-bd15-3cb30ba20e67 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:08:12.192823 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621e18b-1fbc-4d0f-bd15-3cb30ba20e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:08:12.252244 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3621e18b-1fbc-4d0f-bd15-3cb30ba20e67 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:08:12.253457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3621e18b-1fbc-4d0f-bd15-3cb30ba20e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:08:12.333129 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:08:12.345387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 600/1185] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:08:12.355353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f28c30-e0c4-4dd2-8e0a-160d4a9d3b2a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:08:12.356562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-50f28c30-e0c4-4dd2-8e0a-160d4a9d3b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:08:12.457822 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f28c30-e0c4-4dd2-8e0a-160d4a9d3b2a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:08:12.511435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:08:12.534088 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb056be-160f-4b15-9af1-f169c5fbb080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:08:12.534701 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb056be-160f-4b15-9af1-f169c5fbb080 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:08:12.535016 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb056be-160f-4b15-9af1-f169c5fbb080 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:08:12.535308 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb056be-160f-4b15-9af1-f169c5fbb080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:08:12.536875 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3cb056be-160f-4b15-9af1-f169c5fbb080 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:08:12.645530 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb056be-160f-4b15-9af1-f169c5fbb080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:08:12.646919 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb056be-160f-4b15-9af1-f169c5fbb080 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:08:12.665629 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 586/1186] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:08:13.013165 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621e18b-1fbc-4d0f-bd15-3cb30ba20e67 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:08:13.014381 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3621e18b-1fbc-4d0f-bd15-3cb30ba20e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:08:13.026660 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621e18b-1fbc-4d0f-bd15-3cb30ba20e67 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:08:13.121823 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcd1e8-4466-47c2-9aa1-4d5500df36dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:08:13.122449 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcd1e8-4466-47c2-9aa1-4d5500df36dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:08:13.122724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcd1e8-4466-47c2-9aa1-4d5500df36dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:08:13.123025 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcd1e8-4466-47c2-9aa1-4d5500df36dc admin admin] PATH_INFO: `/v3/projects/b277e196181a4f889070a47154446238` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:08:13.133297 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcd1e8-4466-47c2-9aa1-4d5500df36dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b277e196181a4f889070a47154446238)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:08:13.134283 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-e8dcd1e8-4466-47c2-9aa1-4d5500df36dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:08:13.134283 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Oct 17 03:08:13.134283 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: Oct 17 03:08:13.135536 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcd1e8-4466-47c2-9aa1-4d5500df36dc admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:08:13.193092 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e8dcd1e8-4466-47c2-9aa1-4d5500df36dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b277e196181a4f889070a47154446238', 'request_id': 'req-e8dcd1e8-4466-47c2-9aa1-4d5500df36dc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:08:13.199036 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e8dcd1e8-4466-47c2-9aa1-4d5500df36dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b277e196181a4f889070a47154446238', 'request_id': 'req-e8dcd1e8-4466-47c2-9aa1-4d5500df36dc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:08:13.212471 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e8dcd1e8-4466-47c2-9aa1-4d5500df36dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b277e196181a4f889070a47154446238', 'request_id': 'req-e8dcd1e8-4466-47c2-9aa1-4d5500df36dc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:08:13.240802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e8dcd1e8-4466-47c2-9aa1-4d5500df36dc admin admin] The token cache is being invalidate because project b277e196181a4f889070a47154446238 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:08:13.241098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e8dcd1e8-4466-47c2-9aa1-4d5500df36dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:08:13.241531 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e8dcd1e8-4466-47c2-9aa1-4d5500df36dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:08:13.242970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 601/1187] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:08:13 2022] DELETE /identity/v3/projects/b277e196181a4f889070a47154446238 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:08:22.663280 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=62 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:08:23.238732 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:09:41.190509 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee95ea2-a9bb-4bb5-80f2-a3fd9df963a5 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:09:41.191139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee95ea2-a9bb-4bb5-80f2-a3fd9df963a5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:09:41.191139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee95ea2-a9bb-4bb5-80f2-a3fd9df963a5 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:09:41.192438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 587/1188] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:09:41.197443 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d248821-2be2-47cc-abbb-8dc092175468 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:09:41.197872 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d248821-2be2-47cc-abbb-8dc092175468 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:09:41.198006 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d248821-2be2-47cc-abbb-8dc092175468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:09:41.278960 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9d248821-2be2-47cc-abbb-8dc092175468 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:09:41.280089 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9d248821-2be2-47cc-abbb-8dc092175468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:09:41.382276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 602/1189] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:09:41.539370 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee95ea2-a9bb-4bb5-80f2-a3fd9df963a5 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:09:41.541478 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bee95ea2-a9bb-4bb5-80f2-a3fd9df963a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:09:41.660081 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee95ea2-a9bb-4bb5-80f2-a3fd9df963a5 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:09:41.666511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756c6f6-19c9-4199-8d18-b94fd9f9eef9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:09:41.667214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756c6f6-19c9-4199-8d18-b94fd9f9eef9 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:09:41.667499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756c6f6-19c9-4199-8d18-b94fd9f9eef9 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:09:41.667773 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756c6f6-19c9-4199-8d18-b94fd9f9eef9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:09:41.675711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a756c6f6-19c9-4199-8d18-b94fd9f9eef9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:09:41.677386 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a756c6f6-19c9-4199-8d18-b94fd9f9eef9 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:09:41.686844 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 588/1190] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:09:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:09:42.771185 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9dd24-7e54-4d04-a7b0-757d60587999 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:09:42.771722 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9dd24-7e54-4d04-a7b0-757d60587999 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:09:42.771722 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9dd24-7e54-4d04-a7b0-757d60587999 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:09:42.773241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 603/1191] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:09:42.779356 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78620679-0f5c-4479-964b-37325c349675 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:09:42.779767 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78620679-0f5c-4479-964b-37325c349675 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:09:42.779917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78620679-0f5c-4479-964b-37325c349675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:09:42.800335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-78620679-0f5c-4479-964b-37325c349675 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:09:42.801584 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-78620679-0f5c-4479-964b-37325c349675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:09:42.810421 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 589/1192] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:09:42.962408 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9dd24-7e54-4d04-a7b0-757d60587999 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:09:42.977186 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9dd24-7e54-4d04-a7b0-757d60587999 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:09:42.982471 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94014a06-09cd-4ed0-8023-54264ef049e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:09:42.983095 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94014a06-09cd-4ed0-8023-54264ef049e1 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:09:42.983382 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94014a06-09cd-4ed0-8023-54264ef049e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:09:42.983648 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94014a06-09cd-4ed0-8023-54264ef049e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:09:42.991394 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94014a06-09cd-4ed0-8023-54264ef049e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:09:42.992973 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94014a06-09cd-4ed0-8023-54264ef049e1 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:09:43.001981 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 604/1193] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:09:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:09:43.128974 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520a163-d832-4042-87e6-6d5fa6640b58 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:09:43.129265 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520a163-d832-4042-87e6-6d5fa6640b58 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:09:43.129549 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520a163-d832-4042-87e6-6d5fa6640b58 None None] PATH_INFO: `/v3` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:09:43.131010 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 590/1194] 10.222.0.23 () {54 vars in 925 bytes} [Mon Oct 17 03:09:43 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:09:43.144702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f407fe-44ca-4cdb-bed9-35e26ca181c3 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:09:43.144993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f407fe-44ca-4cdb-bed9-35e26ca181c3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:09:43.145273 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f407fe-44ca-4cdb-bed9-35e26ca181c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:09:43.206662 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-29f407fe-44ca-4cdb-bed9-35e26ca181c3 None None] MFA Rules not processed for user `4cc2655cc32e4d63a7000ef41bcbcfea`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:09:43.207934 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-29f407fe-44ca-4cdb-bed9-35e26ca181c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:09:43.225466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:09:43.280423 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 605/1195] 10.222.0.23 () {58 vars in 1015 bytes} [Mon Oct 17 03:09:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:09:46.842175 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba4060-c5bf-40c4-9f2b-4f512ac3973f None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:09:46.842610 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba4060-c5bf-40c4-9f2b-4f512ac3973f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:09:46.842729 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba4060-c5bf-40c4-9f2b-4f512ac3973f None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:09:46.844091 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 591/1196] 10.222.0.27 () {54 vars in 961 bytes} [Mon Oct 17 03:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:09:46.849808 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b01998-47de-4fbb-b4b6-e6388ebcbc88 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:09:46.850231 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b01998-47de-4fbb-b4b6-e6388ebcbc88 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:09:46.850361 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b01998-47de-4fbb-b4b6-e6388ebcbc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:09:46.869511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-40b01998-47de-4fbb-b4b6-e6388ebcbc88 None None] MFA Rules not processed for user `4cc2655cc32e4d63a7000ef41bcbcfea`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:09:46.870683 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-40b01998-47de-4fbb-b4b6-e6388ebcbc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:09:46.879039 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 606/1197] 10.222.0.27 () {58 vars in 1058 bytes} [Mon Oct 17 03:09:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:09:46.886405 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba4060-c5bf-40c4-9f2b-4f512ac3973f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:09:46.900761 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba4060-c5bf-40c4-9f2b-4f512ac3973f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:09:46.905706 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d0159-147a-4812-be82-b8343f6fcf4e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cc2655cc32e4d63a7000ef41bcbcfea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:09:46.906302 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d0159-147a-4812-be82-b8343f6fcf4e service placement] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:09:46.906582 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d0159-147a-4812-be82-b8343f6fcf4e service placement] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:09:46.906850 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d0159-147a-4812-be82-b8343f6fcf4e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:09:46.915652 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d0159-147a-4812-be82-b8343f6fcf4e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:09:46.917221 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d0159-147a-4812-be82-b8343f6fcf4e service placement] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:09:46.926745 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 592/1198] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:09:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:09:54.481772 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8a438-f651-45c6-8ddf-448f826c98e4 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:09:54.482316 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8a438-f651-45c6-8ddf-448f826c98e4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:09:54.482316 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8a438-f651-45c6-8ddf-448f826c98e4 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:09:54.483630 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 607/1199] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:09:54.490335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53010e-51e4-4cbf-84cd-58daafbc0558 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:09:54.490757 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53010e-51e4-4cbf-84cd-58daafbc0558 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:09:54.490942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53010e-51e4-4cbf-84cd-58daafbc0558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:09:54.510362 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8d53010e-51e4-4cbf-84cd-58daafbc0558 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:09:54.511585 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8d53010e-51e4-4cbf-84cd-58daafbc0558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:09:54.519348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 593/1200] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:09:54.674986 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8a438-f651-45c6-8ddf-448f826c98e4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:09:54.690488 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8a438-f651-45c6-8ddf-448f826c98e4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:09:54.696276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a76692-1d8d-4761-940e-16fc469deed0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:09:54.696877 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a76692-1d8d-4761-940e-16fc469deed0 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:09:54.697157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a76692-1d8d-4761-940e-16fc469deed0 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:09:54.697428 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a76692-1d8d-4761-940e-16fc469deed0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:09:54.705694 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a76692-1d8d-4761-940e-16fc469deed0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:09:54.707302 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a76692-1d8d-4761-940e-16fc469deed0 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:09:54.716472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 608/1201] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:09:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:09:55.776410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6f30f-5ed6-4a04-b82e-2d2ffc64cef7 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:09:55.776954 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6f30f-5ed6-4a04-b82e-2d2ffc64cef7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:09:55.776954 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6f30f-5ed6-4a04-b82e-2d2ffc64cef7 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:09:55.778328 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 594/1202] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:09:55.784516 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b67a8-34b2-4444-b2a5-9e31f74f5917 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:09:55.784995 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b67a8-34b2-4444-b2a5-9e31f74f5917 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:09:55.785102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b67a8-34b2-4444-b2a5-9e31f74f5917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:09:55.804683 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-713b67a8-34b2-4444-b2a5-9e31f74f5917 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:09:55.805860 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-713b67a8-34b2-4444-b2a5-9e31f74f5917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:09:55.813681 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 609/1203] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:09:55.964390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6f30f-5ed6-4a04-b82e-2d2ffc64cef7 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:09:55.979373 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6f30f-5ed6-4a04-b82e-2d2ffc64cef7 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:09:55.984711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a3d71-b31e-4f67-aa88-5cb2a7e8ac10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:09:55.985297 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a3d71-b31e-4f67-aa88-5cb2a7e8ac10 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:09:55.985573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a3d71-b31e-4f67-aa88-5cb2a7e8ac10 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:09:55.985842 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a3d71-b31e-4f67-aa88-5cb2a7e8ac10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:09:55.994130 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a3d71-b31e-4f67-aa88-5cb2a7e8ac10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:09:55.995735 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a3d71-b31e-4f67-aa88-5cb2a7e8ac10 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:09:56.005014 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 595/1204] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:09:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:10:00.006403 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47ab63-39ea-4821-9a8f-5e0faa0873c0 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:10:00.006983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47ab63-39ea-4821-9a8f-5e0faa0873c0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:10:00.006983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47ab63-39ea-4821-9a8f-5e0faa0873c0 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:10:00.008298 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 610/1205] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:10:00.013557 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d7f1a-d99d-4ed7-88ee-f9bb3297927e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:10:00.014000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d7f1a-d99d-4ed7-88ee-f9bb3297927e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:10:00.014098 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d7f1a-d99d-4ed7-88ee-f9bb3297927e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:10:00.033694 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d63d7f1a-d99d-4ed7-88ee-f9bb3297927e None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:10:00.034910 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d63d7f1a-d99d-4ed7-88ee-f9bb3297927e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:10:00.043529 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 596/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:10:00.333402 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47ab63-39ea-4821-9a8f-5e0faa0873c0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:10:00.335200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9a47ab63-39ea-4821-9a8f-5e0faa0873c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:10:00.414932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47ab63-39ea-4821-9a8f-5e0faa0873c0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:10:00.418506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb14b8-da80-4dca-8a20-355a761b4b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:10:00.419165 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb14b8-da80-4dca-8a20-355a761b4b52 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:10:00.419449 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb14b8-da80-4dca-8a20-355a761b4b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:10:00.419751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb14b8-da80-4dca-8a20-355a761b4b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:10:00.428921 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb14b8-da80-4dca-8a20-355a761b4b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:10:00.430389 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb14b8-da80-4dca-8a20-355a761b4b52 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:10:00.440285 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 611/1207] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:10:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:10:02.586257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ffd81-3084-414c-9243-271ac8d3d8ec None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:10:02.586890 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ffd81-3084-414c-9243-271ac8d3d8ec None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:10:02.586890 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ffd81-3084-414c-9243-271ac8d3d8ec None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:10:02.588183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 597/1208] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:10:02.593514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3880158-f3f3-4a54-87a0-f1676f3eb642 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:10:02.593966 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3880158-f3f3-4a54-87a0-f1676f3eb642 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:10:02.594063 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3880158-f3f3-4a54-87a0-f1676f3eb642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:10:02.613857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f3880158-f3f3-4a54-87a0-f1676f3eb642 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:10:02.615093 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f3880158-f3f3-4a54-87a0-f1676f3eb642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:10:02.622772 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 612/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:10:02.912924 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ffd81-3084-414c-9243-271ac8d3d8ec None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:10:02.914646 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-673ffd81-3084-414c-9243-271ac8d3d8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:10:02.922165 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=75 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:10:02.929274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ffd81-3084-414c-9243-271ac8d3d8ec None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:10:02.932751 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af2f2d-7a88-4a2f-aab1-a286f64edeab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:10:02.933343 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af2f2d-7a88-4a2f-aab1-a286f64edeab service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:10:02.933627 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af2f2d-7a88-4a2f-aab1-a286f64edeab service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:10:02.933902 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af2f2d-7a88-4a2f-aab1-a286f64edeab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:10:02.942689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af2f2d-7a88-4a2f-aab1-a286f64edeab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:10:02.944159 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af2f2d-7a88-4a2f-aab1-a286f64edeab service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:10:02.953495 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 598/1210] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:10:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:10:06.519577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce7097-6d94-4814-b027-545986570fd5 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:10:06.520102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce7097-6d94-4814-b027-545986570fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:10:06.520102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce7097-6d94-4814-b027-545986570fd5 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:10:06.521519 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 613/1211] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:10:06.527220 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf5c3e-4aea-4544-b3b8-48d548d81a07 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:10:06.527664 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf5c3e-4aea-4544-b3b8-48d548d81a07 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:10:06.527762 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf5c3e-4aea-4544-b3b8-48d548d81a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:10:06.547270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-27cf5c3e-4aea-4544-b3b8-48d548d81a07 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:10:06.548461 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-27cf5c3e-4aea-4544-b3b8-48d548d81a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:10:06.556726 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 599/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:10:06.859897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce7097-6d94-4814-b027-545986570fd5 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:10:06.861546 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0fce7097-6d94-4814-b027-545986570fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:10:06.927239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:10:06.940890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce7097-6d94-4814-b027-545986570fd5 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:10:06.944633 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63c2e5-21cc-41d6-bdbb-41039de7e0bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:10:06.945242 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63c2e5-21cc-41d6-bdbb-41039de7e0bc service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:10:06.945526 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63c2e5-21cc-41d6-bdbb-41039de7e0bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:10:06.945804 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63c2e5-21cc-41d6-bdbb-41039de7e0bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:10:06.954582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63c2e5-21cc-41d6-bdbb-41039de7e0bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:10:06.956050 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63c2e5-21cc-41d6-bdbb-41039de7e0bc service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:10:06.965863 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 614/1213] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:10:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:10:09.098289 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7de67-09f0-40f0-9dc9-574e753c5172 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:10:09.099076 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7de67-09f0-40f0-9dc9-574e753c5172 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:10:09.099366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7de67-09f0-40f0-9dc9-574e753c5172 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:10:09.100724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 600/1214] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:10:09.106064 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46a499-8576-487c-a568-91a41a2e8bfb None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:10:09.106474 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46a499-8576-487c-a568-91a41a2e8bfb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:10:09.106621 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46a499-8576-487c-a568-91a41a2e8bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:10:09.126713 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-5c46a499-8576-487c-a568-91a41a2e8bfb None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:10:09.127951 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5c46a499-8576-487c-a568-91a41a2e8bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:10:09.136312 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 615/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:10:09.436982 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7de67-09f0-40f0-9dc9-574e753c5172 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:10:09.438718 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-62f7de67-09f0-40f0-9dc9-574e753c5172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:10:09.452535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7de67-09f0-40f0-9dc9-574e753c5172 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:10:09.456088 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6a13c-68cd-42a0-aae9-dd9384b9e7b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:10:09.456678 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6a13c-68cd-42a0-aae9-dd9384b9e7b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:10:09.456952 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6a13c-68cd-42a0-aae9-dd9384b9e7b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:10:09.457217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6a13c-68cd-42a0-aae9-dd9384b9e7b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:10:09.466283 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6a13c-68cd-42a0-aae9-dd9384b9e7b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:10:09.467758 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6a13c-68cd-42a0-aae9-dd9384b9e7b8 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:10:09.476986 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 601/1216] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:10:19.121888 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=11 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:10:19.472858 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=14 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:34.926643 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98de9a-0c79-4e36-8218-1eb320849e15 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:34.927904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98de9a-0c79-4e36-8218-1eb320849e15 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:34.928477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98de9a-0c79-4e36-8218-1eb320849e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:34.977835 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8d98de9a-0c79-4e36-8218-1eb320849e15 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:34.979480 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8d98de9a-0c79-4e36-8218-1eb320849e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:34.984107 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3e578-c76f-4140-8c79-5242031a1681 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:34.985168 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3e578-c76f-4140-8c79-5242031a1681 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:34.985671 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3e578-c76f-4140-8c79-5242031a1681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:34.994898 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 616/1217] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:35.002306 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd320bf-482f-409f-b4b1-79eddec01fd8 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.002637 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd320bf-482f-409f-b4b1-79eddec01fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.002888 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd320bf-482f-409f-b4b1-79eddec01fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.028086 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ffe3e578-c76f-4140-8c79-5242031a1681 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:35.029840 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ffe3e578-c76f-4140-8c79-5242031a1681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:35.030651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4bd320bf-482f-409f-b4b1-79eddec01fd8 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:35.032043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4bd320bf-482f-409f-b4b1-79eddec01fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:35.048918 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 602/1218] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:35.051694 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 617/1219] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:11:35.056628 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3e578-c76f-4140-8c79-5242031a1681 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.059122 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832483ae-17c5-4258-af83-da4d79d22ae4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.059413 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832483ae-17c5-4258-af83-da4d79d22ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.059694 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832483ae-17c5-4258-af83-da4d79d22ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.073549 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3e578-c76f-4140-8c79-5242031a1681 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.087789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09846f-accd-47e8-8403-359c93d77a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.089190 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09846f-accd-47e8-8403-359c93d77a68 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.089692 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09846f-accd-47e8-8403-359c93d77a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.090188 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09846f-accd-47e8-8403-359c93d77a68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.091273 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09846f-accd-47e8-8403-359c93d77a68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:11:35.091790 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09846f-accd-47e8-8403-359c93d77a68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.093043 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-1d09846f-accd-47e8-8403-359c93d77a68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:11:35.093043 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Oct 17 03:11:35.093043 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: Oct 17 03:11:35.094330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09846f-accd-47e8-8403-359c93d77a68 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.094746 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-832483ae-17c5-4258-af83-da4d79d22ae4 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:35.096016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-832483ae-17c5-4258-af83-da4d79d22ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:35.112354 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 603/1220] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:11:35.114947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 618/1221] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:35.119844 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09846f-accd-47e8-8403-359c93d77a68 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.123806 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832483ae-17c5-4258-af83-da4d79d22ae4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.135689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09846f-accd-47e8-8403-359c93d77a68 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.146794 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde8913-49e0-41b3-b9eb-944c68ee31dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.147997 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde8913-49e0-41b3-b9eb-944c68ee31dc admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.149103 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde8913-49e0-41b3-b9eb-944c68ee31dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.149861 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde8913-49e0-41b3-b9eb-944c68ee31dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.151040 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde8913-49e0-41b3-b9eb-944c68ee31dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:11:35.151239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832483ae-17c5-4258-af83-da4d79d22ae4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.151766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde8913-49e0-41b3-b9eb-944c68ee31dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.153615 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde8913-49e0-41b3-b9eb-944c68ee31dc admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.169423 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6954127-f6bd-4cea-afd5-128f296447d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.170069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6954127-f6bd-4cea-afd5-128f296447d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.170356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6954127-f6bd-4cea-afd5-128f296447d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.170637 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6954127-f6bd-4cea-afd5-128f296447d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.171801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6954127-f6bd-4cea-afd5-128f296447d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.172483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-d6954127-f6bd-4cea-afd5-128f296447d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:11:35.172483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Oct 17 03:11:35.172483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: Oct 17 03:11:35.173689 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6954127-f6bd-4cea-afd5-128f296447d8 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.176209 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 604/1222] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:11:35.184658 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde8913-49e0-41b3-b9eb-944c68ee31dc admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.199029 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde8913-49e0-41b3-b9eb-944c68ee31dc admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.219319 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b2161-1b13-4cc0-b93a-26de561d25e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.219944 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b2161-1b13-4cc0-b93a-26de561d25e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.220225 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b2161-1b13-4cc0-b93a-26de561d25e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.220498 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b2161-1b13-4cc0-b93a-26de561d25e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.227564 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b2161-1b13-4cc0-b93a-26de561d25e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.230485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b2161-1b13-4cc0-b93a-26de561d25e0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.309196 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 605/1223] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:11:35.311107 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9df6ab-8616-4d64-b9b1-7aea0fc04db1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.311405 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9df6ab-8616-4d64-b9b1-7aea0fc04db1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.311682 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9df6ab-8616-4d64-b9b1-7aea0fc04db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.315047 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 619/1224] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/projects => generated 377 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:35.316557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f832b2-c36d-437d-b3fc-80146bb0f9a9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.319570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f832b2-c36d-437d-b3fc-80146bb0f9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.320588 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f832b2-c36d-437d-b3fc-80146bb0f9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.347909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-cc9df6ab-8616-4d64-b9b1-7aea0fc04db1 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:35.349523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cc9df6ab-8616-4d64-b9b1-7aea0fc04db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:35.356359 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-35f832b2-c36d-437d-b3fc-80146bb0f9a9 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:35.357877 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-35f832b2-c36d-437d-b3fc-80146bb0f9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:35.415531 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 606/1225] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:35.416777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b462e-41da-4c1a-9c47-539c2514d382 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.417061 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b462e-41da-4c1a-9c47-539c2514d382 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.417333 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b462e-41da-4c1a-9c47-539c2514d382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.427976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 620/1226] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:35.429399 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f832b2-c36d-437d-b3fc-80146bb0f9a9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.440997 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f832b2-c36d-437d-b3fc-80146bb0f9a9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.443218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2c5b462e-41da-4c1a-9c47-539c2514d382 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:35.444486 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2c5b462e-41da-4c1a-9c47-539c2514d382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:35.445953 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67023272-d3ba-4cb6-80da-68fd10d66f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.446563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67023272-d3ba-4cb6-80da-68fd10d66f1f admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.446844 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67023272-d3ba-4cb6-80da-68fd10d66f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.447149 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67023272-d3ba-4cb6-80da-68fd10d66f1f admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.447788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67023272-d3ba-4cb6-80da-68fd10d66f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.449741 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67023272-d3ba-4cb6-80da-68fd10d66f1f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.453265 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 607/1227] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:35.453866 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b462e-41da-4c1a-9c47-539c2514d382 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.468091 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b462e-41da-4c1a-9c47-539c2514d382 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.475464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 621/1228] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/users => generated 294 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:35.475623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3add8-a835-4c16-bb6d-db6679a6579e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.475623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3add8-a835-4c16-bb6d-db6679a6579e admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.475623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3add8-a835-4c16-bb6d-db6679a6579e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.475623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3add8-a835-4c16-bb6d-db6679a6579e admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.476106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfbe66-f97b-4b34-a133-9f5d930d007b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.476106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfbe66-f97b-4b34-a133-9f5d930d007b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.476106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfbe66-f97b-4b34-a133-9f5d930d007b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.476500 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3add8-a835-4c16-bb6d-db6679a6579e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.476500 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-81a3add8-a835-4c16-bb6d-db6679a6579e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:11:35.476500 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Oct 17 03:11:35.476500 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: Oct 17 03:11:35.477322 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3add8-a835-4c16-bb6d-db6679a6579e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.495296 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-82cfbe66-f97b-4b34-a133-9f5d930d007b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:35.496772 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-82cfbe66-f97b-4b34-a133-9f5d930d007b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:35.501837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 608/1229] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/users => generated 289 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:35.503227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298d416-2d26-4fb5-83cd-52a041de63e6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.503507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298d416-2d26-4fb5-83cd-52a041de63e6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.503782 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7298d416-2d26-4fb5-83cd-52a041de63e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.504934 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 622/1230] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:35.512162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ee538-cb78-4ee8-af0a-4dccae49e783 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.512566 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ee538-cb78-4ee8-af0a-4dccae49e783 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.512761 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ee538-cb78-4ee8-af0a-4dccae49e783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.525701 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-7298d416-2d26-4fb5-83cd-52a041de63e6 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:35.526934 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7298d416-2d26-4fb5-83cd-52a041de63e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:35.533221 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-139ee538-cb78-4ee8-af0a-4dccae49e783 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:35.534389 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-139ee538-cb78-4ee8-af0a-4dccae49e783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:35.534852 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 609/1231] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:35.540159 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ebabc-578c-4217-ba8c-0e2a48cacf89 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.540447 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ebabc-578c-4217-ba8c-0e2a48cacf89 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.540745 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ebabc-578c-4217-ba8c-0e2a48cacf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.542993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 623/1232] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:35.544370 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3ed47-d331-48bb-98b2-a42049f24041 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.544664 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3ed47-d331-48bb-98b2-a42049f24041 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.544942 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3ed47-d331-48bb-98b2-a42049f24041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.565423 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-45f3ed47-d331-48bb-98b2-a42049f24041 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:35.566621 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-45f3ed47-d331-48bb-98b2-a42049f24041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:35.570408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-099ebabc-578c-4217-ba8c-0e2a48cacf89 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:35.571616 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-099ebabc-578c-4217-ba8c-0e2a48cacf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:35.575290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 624/1233] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:35.576414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d526bf0-5846-40b0-bb94-745ea3812a50 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.576703 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d526bf0-5846-40b0-bb94-745ea3812a50 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.576975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d526bf0-5846-40b0-bb94-745ea3812a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.580053 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 610/1234] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:35.580834 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ebabc-578c-4217-ba8c-0e2a48cacf89 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.590495 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ebabc-578c-4217-ba8c-0e2a48cacf89 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.595413 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eda33c-a4f2-401e-9a29-d022bcf42002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.596003 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eda33c-a4f2-401e-9a29-d022bcf42002 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.596278 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eda33c-a4f2-401e-9a29-d022bcf42002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.596547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eda33c-a4f2-401e-9a29-d022bcf42002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.596698 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1d526bf0-5846-40b0-bb94-745ea3812a50 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:35.597180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eda33c-a4f2-401e-9a29-d022bcf42002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.597885 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1d526bf0-5846-40b0-bb94-745ea3812a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:35.598782 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eda33c-a4f2-401e-9a29-d022bcf42002 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.605918 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 625/1235] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:35.606665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d526bf0-5846-40b0-bb94-745ea3812a50 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.612742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 611/1236] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:35.614200 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104a499-9912-4c0d-a72c-13e8c4bf5599 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.614555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104a499-9912-4c0d-a72c-13e8c4bf5599 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.614841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104a499-9912-4c0d-a72c-13e8c4bf5599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.618472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d526bf0-5846-40b0-bb94-745ea3812a50 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.622952 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257249a8-f62e-47b2-a944-8f8410a6c95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.623562 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257249a8-f62e-47b2-a944-8f8410a6c95d admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.623834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257249a8-f62e-47b2-a944-8f8410a6c95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.624110 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257249a8-f62e-47b2-a944-8f8410a6c95d admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.624751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257249a8-f62e-47b2-a944-8f8410a6c95d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.626376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257249a8-f62e-47b2-a944-8f8410a6c95d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.635966 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e104a499-9912-4c0d-a72c-13e8c4bf5599 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:35.636456 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 626/1237] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:35.637153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e104a499-9912-4c0d-a72c-13e8c4bf5599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:35.637699 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e458b89-bec2-47de-acb2-ec41e62e148f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.638002 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e458b89-bec2-47de-acb2-ec41e62e148f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.638284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e458b89-bec2-47de-acb2-ec41e62e148f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.645546 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 612/1238] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:35.646709 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c0274-2ee4-4722-9b2c-7964b8576b98 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.647028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c0274-2ee4-4722-9b2c-7964b8576b98 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.647305 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c0274-2ee4-4722-9b2c-7964b8576b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.659631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6e458b89-bec2-47de-acb2-ec41e62e148f None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:35.660824 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6e458b89-bec2-47de-acb2-ec41e62e148f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:35.667087 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a39c0274-2ee4-4722-9b2c-7964b8576b98 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:35.668253 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a39c0274-2ee4-4722-9b2c-7964b8576b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:35.668999 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 627/1239] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:35.669769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e458b89-bec2-47de-acb2-ec41e62e148f None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.679027 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e458b89-bec2-47de-acb2-ec41e62e148f None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.680416 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 613/1240] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:35.681205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c0274-2ee4-4722-9b2c-7964b8576b98 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.687699 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65556fbe-daff-4086-a6ef-492a7bac2e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.688390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65556fbe-daff-4086-a6ef-492a7bac2e36 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.688678 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65556fbe-daff-4086-a6ef-492a7bac2e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.688952 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65556fbe-daff-4086-a6ef-492a7bac2e36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.689596 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65556fbe-daff-4086-a6ef-492a7bac2e36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:11:35.689906 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65556fbe-daff-4086-a6ef-492a7bac2e36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.690571 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c0274-2ee4-4722-9b2c-7964b8576b98 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.691594 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65556fbe-daff-4086-a6ef-492a7bac2e36 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.695430 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1712e-16a4-4f4c-9c12-15474355a353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.696017 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1712e-16a4-4f4c-9c12-15474355a353 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.696301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1712e-16a4-4f4c-9c12-15474355a353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.696578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c1712e-16a4-4f4c-9c12-15474355a353 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.697248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c1712e-16a4-4f4c-9c12-15474355a353 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:11:35.697557 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c1712e-16a4-4f4c-9c12-15474355a353 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.699167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c1712e-16a4-4f4c-9c12-15474355a353 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.703523 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 628/1241] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:11:35.704549 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65556fbe-daff-4086-a6ef-492a7bac2e36 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.711560 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 614/1242] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:11:35.712446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1712e-16a4-4f4c-9c12-15474355a353 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.715602 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65556fbe-daff-4086-a6ef-492a7bac2e36 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.720340 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c531a0d-0b97-44e4-b91b-e5f34de96a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.720940 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c531a0d-0b97-44e4-b91b-e5f34de96a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.721214 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c531a0d-0b97-44e4-b91b-e5f34de96a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.721489 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c531a0d-0b97-44e4-b91b-e5f34de96a00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.722202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c531a0d-0b97-44e4-b91b-e5f34de96a00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:11:35.722500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c531a0d-0b97-44e4-b91b-e5f34de96a00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.723514 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c1712e-16a4-4f4c-9c12-15474355a353 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.724114 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c531a0d-0b97-44e4-b91b-e5f34de96a00 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.728830 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801f8c4-e311-452b-b041-cbbbf88b0689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.729399 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801f8c4-e311-452b-b041-cbbbf88b0689 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.729682 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801f8c4-e311-452b-b041-cbbbf88b0689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.729956 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801f8c4-e311-452b-b041-cbbbf88b0689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.730597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801f8c4-e311-452b-b041-cbbbf88b0689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.732217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801f8c4-e311-452b-b041-cbbbf88b0689 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.737477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 629/1243] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:11:35.738268 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c531a0d-0b97-44e4-b91b-e5f34de96a00 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.749116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c531a0d-0b97-44e4-b91b-e5f34de96a00 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.749367 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 615/1244] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:35.750173 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801f8c4-e311-452b-b041-cbbbf88b0689 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.755577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff41fce-7f10-4dfa-8c19-41d99b272e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.756267 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff41fce-7f10-4dfa-8c19-41d99b272e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.756559 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff41fce-7f10-4dfa-8c19-41d99b272e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.756835 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff41fce-7f10-4dfa-8c19-41d99b272e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.758250 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff41fce-7f10-4dfa-8c19-41d99b272e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.759888 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff41fce-7f10-4dfa-8c19-41d99b272e4f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.760080 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801f8c4-e311-452b-b041-cbbbf88b0689 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.764467 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9119e-e24d-480e-871c-1bae90706793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.765564 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9119e-e24d-480e-871c-1bae90706793 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.765847 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9119e-e24d-480e-871c-1bae90706793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.766117 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9119e-e24d-480e-871c-1bae90706793 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.766767 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b9119e-e24d-480e-871c-1bae90706793 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:11:35.767076 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b9119e-e24d-480e-871c-1bae90706793 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.768634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b9119e-e24d-480e-871c-1bae90706793 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.773446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 630/1245] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:35.773446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff41fce-7f10-4dfa-8c19-41d99b272e4f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.780456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 616/1246] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:11:35.781253 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9119e-e24d-480e-871c-1bae90706793 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.782995 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff41fce-7f10-4dfa-8c19-41d99b272e4f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.787653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b47113-88a5-47ae-8387-051b57524041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.788255 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b47113-88a5-47ae-8387-051b57524041 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.789188 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b47113-88a5-47ae-8387-051b57524041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.789459 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b47113-88a5-47ae-8387-051b57524041 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.790219 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b47113-88a5-47ae-8387-051b57524041 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:11:35.790529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b47113-88a5-47ae-8387-051b57524041 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.792155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b47113-88a5-47ae-8387-051b57524041 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.793119 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9119e-e24d-480e-871c-1bae90706793 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.797820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81025371-5589-4a43-bb7b-4e6eb5c60c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.798401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81025371-5589-4a43-bb7b-4e6eb5c60c51 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.799129 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81025371-5589-4a43-bb7b-4e6eb5c60c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.799268 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:35.799358 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81025371-5589-4a43-bb7b-4e6eb5c60c51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.799613 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81025371-5589-4a43-bb7b-4e6eb5c60c51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:11:35.799909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81025371-5589-4a43-bb7b-4e6eb5c60c51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.801871 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81025371-5589-4a43-bb7b-4e6eb5c60c51 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.803478 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:35.805301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 631/1247] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:11:35.815167 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b47113-88a5-47ae-8387-051b57524041 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.815336 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 617/1248] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:11:35.816058 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81025371-5589-4a43-bb7b-4e6eb5c60c51 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.827066 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b47113-88a5-47ae-8387-051b57524041 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.828085 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81025371-5589-4a43-bb7b-4e6eb5c60c51 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.831812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7537a9e-ee55-4956-895d-4a4f83f3c59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.832485 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7537a9e-ee55-4956-895d-4a4f83f3c59d admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.832628 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b476b78-ab7a-43bc-bcfd-0f1308225952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.832769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7537a9e-ee55-4956-895d-4a4f83f3c59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.833035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7537a9e-ee55-4956-895d-4a4f83f3c59d admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.833219 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b476b78-ab7a-43bc-bcfd-0f1308225952 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.833500 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b476b78-ab7a-43bc-bcfd-0f1308225952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.833669 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7537a9e-ee55-4956-895d-4a4f83f3c59d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.833771 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b476b78-ab7a-43bc-bcfd-0f1308225952 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.834386 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b476b78-ab7a-43bc-bcfd-0f1308225952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.835755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7537a9e-ee55-4956-895d-4a4f83f3c59d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.836420 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b476b78-ab7a-43bc-bcfd-0f1308225952 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.863349 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 618/1250] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:35.863555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 632/1250] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:35.866893 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b476b78-ab7a-43bc-bcfd-0f1308225952 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.867823 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7537a9e-ee55-4956-895d-4a4f83f3c59d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.928635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b476b78-ab7a-43bc-bcfd-0f1308225952 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.930727 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7537a9e-ee55-4956-895d-4a4f83f3c59d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:35.934325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa4525-23e7-4caa-a519-555986cbada6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.935075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa4525-23e7-4caa-a519-555986cbada6 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.935337 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa4525-23e7-4caa-a519-555986cbada6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.935666 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa4525-23e7-4caa-a519-555986cbada6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.936498 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa4525-23e7-4caa-a519-555986cbada6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.936625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e49a29-d258-4c48-bcd6-8a581219daad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:35.937552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e49a29-d258-4c48-bcd6-8a581219daad admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:35.937552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e49a29-d258-4c48-bcd6-8a581219daad admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:35.937813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e49a29-d258-4c48-bcd6-8a581219daad admin admin] PATH_INFO: `/v3/projects/36f63b28bb4f4654b53a01507b337d52/users/37e6d36d529146299970874660e92e43/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:35.939164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa4525-23e7-4caa-a519-555986cbada6 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.957079 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e49a29-d258-4c48-bcd6-8a581219daad admin admin] RBAC: Authorizing `identity:create_grant(project_id=36f63b28bb4f4654b53a01507b337d52, user_id=37e6d36d529146299970874660e92e43, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:35.959750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e49a29-d258-4c48-bcd6-8a581219daad admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:35.973586 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 619/1251] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:35.974550 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 633/1252] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:35 2022] PUT /identity/v3/projects/36f63b28bb4f4654b53a01507b337d52/users/37e6d36d529146299970874660e92e43/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:35.974874 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa4525-23e7-4caa-a519-555986cbada6 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:35.975594 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e49a29-d258-4c48-bcd6-8a581219daad admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.039278 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e49a29-d258-4c48-bcd6-8a581219daad admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.043202 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa4525-23e7-4caa-a519-555986cbada6 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.044081 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6530898-5bb4-4313-b479-802efd17a646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.044698 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6530898-5bb4-4313-b479-802efd17a646 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.045022 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6530898-5bb4-4313-b479-802efd17a646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.045277 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6530898-5bb4-4313-b479-802efd17a646 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.045950 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6530898-5bb4-4313-b479-802efd17a646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.047860 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6530898-5bb4-4313-b479-802efd17a646 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.047983 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3642a-c0b4-4586-833f-239b3e1ece5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.048532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3642a-c0b4-4586-833f-239b3e1ece5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.048815 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3642a-c0b4-4586-833f-239b3e1ece5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.049096 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3642a-c0b4-4586-833f-239b3e1ece5d admin admin] PATH_INFO: `/v3/projects/e5bad80dc0d74119b2a9cd5c1935e8d5/users/ce25db99a7b14d0bbaca41fff80e3da9/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.063551 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3642a-c0b4-4586-833f-239b3e1ece5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5bad80dc0d74119b2a9cd5c1935e8d5, user_id=ce25db99a7b14d0bbaca41fff80e3da9, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.066706 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3642a-c0b4-4586-833f-239b3e1ece5d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.079209 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 634/1253] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:11:35 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:36.079209 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6530898-5bb4-4313-b479-802efd17a646 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.081028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 620/1254] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:35 2022] PUT /identity/v3/projects/e5bad80dc0d74119b2a9cd5c1935e8d5/users/ce25db99a7b14d0bbaca41fff80e3da9/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:36.082130 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3642a-c0b4-4586-833f-239b3e1ece5d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.124348 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:36.125315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:36.141991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6530898-5bb4-4313-b479-802efd17a646 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.146596 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b8039-2c5c-419c-952a-2f9ea2ac7a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.147100 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3642a-c0b4-4586-833f-239b3e1ece5d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.147233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b8039-2c5c-419c-952a-2f9ea2ac7a64 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.147503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b8039-2c5c-419c-952a-2f9ea2ac7a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.148330 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b8039-2c5c-419c-952a-2f9ea2ac7a64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.149055 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b8039-2c5c-419c-952a-2f9ea2ac7a64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.151246 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b8039-2c5c-419c-952a-2f9ea2ac7a64 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.152559 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826560e8-5d68-41bc-a3be-aa8e5c828e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.153166 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826560e8-5d68-41bc-a3be-aa8e5c828e32 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.153445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826560e8-5d68-41bc-a3be-aa8e5c828e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.153718 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826560e8-5d68-41bc-a3be-aa8e5c828e32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.154342 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826560e8-5d68-41bc-a3be-aa8e5c828e32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.156278 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826560e8-5d68-41bc-a3be-aa8e5c828e32 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.179469 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 635/1255] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:11:36 2022] POST /identity/v3/projects => generated 403 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:36.181624 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b8039-2c5c-419c-952a-2f9ea2ac7a64 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.182972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 621/1256] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:11:36 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:36.183637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826560e8-5d68-41bc-a3be-aa8e5c828e32 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.247280 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826560e8-5d68-41bc-a3be-aa8e5c828e32 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.247648 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b8039-2c5c-419c-952a-2f9ea2ac7a64 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.251812 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591348d-04a5-444a-98d3-329a527bb304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.252306 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc10938-6e62-4239-a60a-60b8186b4878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.252440 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591348d-04a5-444a-98d3-329a527bb304 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.252688 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591348d-04a5-444a-98d3-329a527bb304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.252912 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc10938-6e62-4239-a60a-60b8186b4878 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.253019 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591348d-04a5-444a-98d3-329a527bb304 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.253195 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc10938-6e62-4239-a60a-60b8186b4878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.253469 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc10938-6e62-4239-a60a-60b8186b4878 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.253630 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591348d-04a5-444a-98d3-329a527bb304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.254107 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc10938-6e62-4239-a60a-60b8186b4878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.255571 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591348d-04a5-444a-98d3-329a527bb304 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.256055 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc10938-6e62-4239-a60a-60b8186b4878 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.282162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 622/1257] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:11:36 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:36.282162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a4235-24ae-4b06-bce7-bb56ea6c6205 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.282162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a4235-24ae-4b06-bce7-bb56ea6c6205 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.282162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a4235-24ae-4b06-bce7-bb56ea6c6205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.283849 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 636/1258] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:11:36 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:36.284645 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc10938-6e62-4239-a60a-60b8186b4878 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.294681 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc10938-6e62-4239-a60a-60b8186b4878 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.299611 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbbdb6-600e-415b-99f5-ce4528030525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.300212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbbdb6-600e-415b-99f5-ce4528030525 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.300494 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbbdb6-600e-415b-99f5-ce4528030525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.300747 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-357a4235-24ae-4b06-bce7-bb56ea6c6205 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:36.300891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbbdb6-600e-415b-99f5-ce4528030525 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.301594 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bbbdb6-600e-415b-99f5-ce4528030525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.301942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-357a4235-24ae-4b06-bce7-bb56ea6c6205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:36.303574 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bbbdb6-600e-415b-99f5-ce4528030525 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.310315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 623/1259] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:36.311083 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a4235-24ae-4b06-bce7-bb56ea6c6205 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.325055 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a4235-24ae-4b06-bce7-bb56ea6c6205 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.329724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 637/1260] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:11:36 2022] POST /identity/v3/users => generated 293 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:36.329724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e888a-01dd-49e7-a251-bec21507c34b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.329724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e888a-01dd-49e7-a251-bec21507c34b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.329724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e888a-01dd-49e7-a251-bec21507c34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.332946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184329c-48c0-46fd-a764-0259ef21ca00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.333551 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184329c-48c0-46fd-a764-0259ef21ca00 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.333830 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184329c-48c0-46fd-a764-0259ef21ca00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.334100 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184329c-48c0-46fd-a764-0259ef21ca00 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.334728 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5184329c-48c0-46fd-a764-0259ef21ca00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.337915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5184329c-48c0-46fd-a764-0259ef21ca00 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.348497 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d21e888a-01dd-49e7-a251-bec21507c34b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:36.349735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d21e888a-01dd-49e7-a251-bec21507c34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:36.359143 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 638/1261] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:36.362369 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e888a-01dd-49e7-a251-bec21507c34b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.366154 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 624/1262] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:11:36 2022] POST /identity/v3/users => generated 287 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:36.366154 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184329c-48c0-46fd-a764-0259ef21ca00 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.370884 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e888a-01dd-49e7-a251-bec21507c34b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.375476 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57895e0c-8e7e-4411-a365-74032223d29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.375753 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184329c-48c0-46fd-a764-0259ef21ca00 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.376074 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57895e0c-8e7e-4411-a365-74032223d29c admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.376359 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57895e0c-8e7e-4411-a365-74032223d29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.376669 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57895e0c-8e7e-4411-a365-74032223d29c admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.377271 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57895e0c-8e7e-4411-a365-74032223d29c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.379268 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57895e0c-8e7e-4411-a365-74032223d29c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.380275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12db78-5e35-4d91-b959-68c546f1e6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.380865 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12db78-5e35-4d91-b959-68c546f1e6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.381144 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12db78-5e35-4d91-b959-68c546f1e6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.381422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12db78-5e35-4d91-b959-68c546f1e6e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.383002 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12db78-5e35-4d91-b959-68c546f1e6e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.384897 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12db78-5e35-4d91-b959-68c546f1e6e5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.403485 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 639/1263] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:11:36 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:36.404426 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57895e0c-8e7e-4411-a365-74032223d29c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.409186 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 625/1264] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:11:36 2022] POST /identity/v3/users => generated 302 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:36.410344 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12db78-5e35-4d91-b959-68c546f1e6e5 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.417394 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57895e0c-8e7e-4411-a365-74032223d29c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.421749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12db78-5e35-4d91-b959-68c546f1e6e5 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.422006 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54b578-679b-4b59-adc9-2444045c84a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.422629 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54b578-679b-4b59-adc9-2444045c84a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.422936 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54b578-679b-4b59-adc9-2444045c84a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.423211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54b578-679b-4b59-adc9-2444045c84a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.423851 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54b578-679b-4b59-adc9-2444045c84a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.425462 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54b578-679b-4b59-adc9-2444045c84a4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.426496 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbcb68-ec87-4137-8ea0-5ea11befdb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.427112 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbcb68-ec87-4137-8ea0-5ea11befdb45 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.427393 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbcb68-ec87-4137-8ea0-5ea11befdb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.427668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbcb68-ec87-4137-8ea0-5ea11befdb45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.428304 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbcb68-ec87-4137-8ea0-5ea11befdb45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.430647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbcb68-ec87-4137-8ea0-5ea11befdb45 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.436425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 640/1265] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:36.437214 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54b578-679b-4b59-adc9-2444045c84a4 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.446804 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 626/1266] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:36.446804 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbcb68-ec87-4137-8ea0-5ea11befdb45 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.455248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbcb68-ec87-4137-8ea0-5ea11befdb45 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.460176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63cd91-2eb5-4ae2-9418-8b95f4977f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.460804 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63cd91-2eb5-4ae2-9418-8b95f4977f8a admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.461081 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63cd91-2eb5-4ae2-9418-8b95f4977f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.461352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63cd91-2eb5-4ae2-9418-8b95f4977f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.461990 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63cd91-2eb5-4ae2-9418-8b95f4977f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.463611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63cd91-2eb5-4ae2-9418-8b95f4977f8a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.473542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 627/1267] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:36.474424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63cd91-2eb5-4ae2-9418-8b95f4977f8a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.485124 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63cd91-2eb5-4ae2-9418-8b95f4977f8a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.489671 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed7ef0-2b00-4c02-bd43-04196568e8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.490314 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed7ef0-2b00-4c02-bd43-04196568e8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.490597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed7ef0-2b00-4c02-bd43-04196568e8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.490886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed7ef0-2b00-4c02-bd43-04196568e8b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.491653 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed7ef0-2b00-4c02-bd43-04196568e8b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.493254 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed7ef0-2b00-4c02-bd43-04196568e8b3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.495844 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54b578-679b-4b59-adc9-2444045c84a4 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.499624 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde6938-b3be-495b-90ee-1f7a3ed9645a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.500175 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde6938-b3be-495b-90ee-1f7a3ed9645a service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.500464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde6938-b3be-495b-90ee-1f7a3ed9645a service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.500738 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde6938-b3be-495b-90ee-1f7a3ed9645a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.503215 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 628/1268] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:36.503990 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed7ef0-2b00-4c02-bd43-04196568e8b3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.511067 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde6938-b3be-495b-90ee-1f7a3ed9645a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.512529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde6938-b3be-495b-90ee-1f7a3ed9645a service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.513619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed7ef0-2b00-4c02-bd43-04196568e8b3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.517257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d00ff-6b61-4645-bc63-990f969221d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.518457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d00ff-6b61-4645-bc63-990f969221d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.518787 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d00ff-6b61-4645-bc63-990f969221d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.519096 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d00ff-6b61-4645-bc63-990f969221d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.522617 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 641/1269] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:36.523378 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde6938-b3be-495b-90ee-1f7a3ed9645a service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.528786 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d00ff-6b61-4645-bc63-990f969221d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.530259 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d00ff-6b61-4645-bc63-990f969221d0 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.533079 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde6938-b3be-495b-90ee-1f7a3ed9645a service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.537468 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eeb51c-6479-4d68-8c7d-c1e0a4da935a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.538066 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eeb51c-6479-4d68-8c7d-c1e0a4da935a admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.538384 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eeb51c-6479-4d68-8c7d-c1e0a4da935a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.539415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eeb51c-6479-4d68-8c7d-c1e0a4da935a admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.540028 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eeb51c-6479-4d68-8c7d-c1e0a4da935a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.540800 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 629/1270] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:36.541462 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d00ff-6b61-4645-bc63-990f969221d0 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.541646 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eeb51c-6479-4d68-8c7d-c1e0a4da935a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.543842 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:36.552245 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d00ff-6b61-4645-bc63-990f969221d0 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.555003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 642/1271] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:36.555851 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eeb51c-6479-4d68-8c7d-c1e0a4da935a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.561315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f4c54-c06d-4dd1-a747-a3c716e6a476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.562011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f4c54-c06d-4dd1-a747-a3c716e6a476 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.562339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f4c54-c06d-4dd1-a747-a3c716e6a476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.562600 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f4c54-c06d-4dd1-a747-a3c716e6a476 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.563359 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f4c54-c06d-4dd1-a747-a3c716e6a476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.565234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f4c54-c06d-4dd1-a747-a3c716e6a476 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.567195 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eeb51c-6479-4d68-8c7d-c1e0a4da935a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.572180 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b5685-d9a7-4e41-a420-7587d8102b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.572788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b5685-d9a7-4e41-a420-7587d8102b1f admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.573065 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b5685-d9a7-4e41-a420-7587d8102b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.573340 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b5685-d9a7-4e41-a420-7587d8102b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.573978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b5685-d9a7-4e41-a420-7587d8102b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.575657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b5685-d9a7-4e41-a420-7587d8102b1f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.578905 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 630/1272] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:36.582225 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f4c54-c06d-4dd1-a747-a3c716e6a476 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.586893 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 643/1273] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:36.587673 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b5685-d9a7-4e41-a420-7587d8102b1f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.594112 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f4c54-c06d-4dd1-a747-a3c716e6a476 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.597692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b5685-d9a7-4e41-a420-7587d8102b1f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.599209 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3898d0-de6c-49cf-b027-88e60d18c8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.599834 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3898d0-de6c-49cf-b027-88e60d18c8cf admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.600249 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3898d0-de6c-49cf-b027-88e60d18c8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.600627 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3898d0-de6c-49cf-b027-88e60d18c8cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.602002 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3898d0-de6c-49cf-b027-88e60d18c8cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.603950 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab196c-4d19-404e-80d9-58ba1bee10cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.604156 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3898d0-de6c-49cf-b027-88e60d18c8cf admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.604516 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab196c-4d19-404e-80d9-58ba1bee10cb admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.604872 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab196c-4d19-404e-80d9-58ba1bee10cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.605121 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab196c-4d19-404e-80d9-58ba1bee10cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.605728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab196c-4d19-404e-80d9-58ba1bee10cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.607696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab196c-4d19-404e-80d9-58ba1bee10cb admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.615908 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 631/1274] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:36.616859 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3898d0-de6c-49cf-b027-88e60d18c8cf admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.623269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 644/1275] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:36.624483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab196c-4d19-404e-80d9-58ba1bee10cb admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.626828 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:36.631079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3898d0-de6c-49cf-b027-88e60d18c8cf admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.636864 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab196c-4d19-404e-80d9-58ba1bee10cb admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.638597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d71ebe-1222-4524-8d49-a41d14890f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.639497 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d71ebe-1222-4524-8d49-a41d14890f6d admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.639924 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d71ebe-1222-4524-8d49-a41d14890f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.640291 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d71ebe-1222-4524-8d49-a41d14890f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.641080 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d71ebe-1222-4524-8d49-a41d14890f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.642299 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97d629-6940-45a2-b67c-85be6a64f00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.642893 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d71ebe-1222-4524-8d49-a41d14890f6d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.643018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97d629-6940-45a2-b67c-85be6a64f00d admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.643235 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97d629-6940-45a2-b67c-85be6a64f00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.643504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97d629-6940-45a2-b67c-85be6a64f00d admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.644135 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97d629-6940-45a2-b67c-85be6a64f00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.646449 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97d629-6940-45a2-b67c-85be6a64f00d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.655466 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 632/1276] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:36.656712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d71ebe-1222-4524-8d49-a41d14890f6d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.659011 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 645/1277] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:36.660111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97d629-6940-45a2-b67c-85be6a64f00d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.669544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d71ebe-1222-4524-8d49-a41d14890f6d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.671707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97d629-6940-45a2-b67c-85be6a64f00d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.675209 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a7ed4-9cc8-4946-ac48-ce6f1ca9a42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.675823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a7ed4-9cc8-4946-ac48-ce6f1ca9a42d admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.676267 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a7ed4-9cc8-4946-ac48-ce6f1ca9a42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.676552 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a7ed4-9cc8-4946-ac48-ce6f1ca9a42d admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.677256 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a7ed4-9cc8-4946-ac48-ce6f1ca9a42d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.677913 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19763bc-8cab-4682-b44d-7a7890f8de87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.678794 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19763bc-8cab-4682-b44d-7a7890f8de87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.678939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a7ed4-9cc8-4946-ac48-ce6f1ca9a42d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.679349 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19763bc-8cab-4682-b44d-7a7890f8de87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.679708 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19763bc-8cab-4682-b44d-7a7890f8de87 admin admin] PATH_INFO: `/v3/projects/ee21a91b84794e80b208fe72b7f1e010/users/9c1787a9ef7c44e28076aaa80c0152fd/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.690337 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 633/1278] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:36.691014 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a7ed4-9cc8-4946-ac48-ce6f1ca9a42d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.701924 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a7ed4-9cc8-4946-ac48-ce6f1ca9a42d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.702101 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19763bc-8cab-4682-b44d-7a7890f8de87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee21a91b84794e80b208fe72b7f1e010, user_id=9c1787a9ef7c44e28076aaa80c0152fd, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.704834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19763bc-8cab-4682-b44d-7a7890f8de87 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.706904 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ac3bf-9fc4-4126-8183-41eadcc21206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.707445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ac3bf-9fc4-4126-8183-41eadcc21206 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.707713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ac3bf-9fc4-4126-8183-41eadcc21206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.707985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ac3bf-9fc4-4126-8183-41eadcc21206 admin admin] PATH_INFO: `/v3/projects/d03f03d21e9e42fb91606306241a3fb7/users/4443729a5a5341b6ad64f6fa131a5a54/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.727039 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 646/1279] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:36 2022] PUT /identity/v3/projects/ee21a91b84794e80b208fe72b7f1e010/users/9c1787a9ef7c44e28076aaa80c0152fd/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:36.727039 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19763bc-8cab-4682-b44d-7a7890f8de87 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.730369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ac3bf-9fc4-4126-8183-41eadcc21206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d03f03d21e9e42fb91606306241a3fb7, user_id=4443729a5a5341b6ad64f6fa131a5a54, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.733209 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ac3bf-9fc4-4126-8183-41eadcc21206 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.749162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 634/1280] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:36 2022] PUT /identity/v3/projects/d03f03d21e9e42fb91606306241a3fb7/users/4443729a5a5341b6ad64f6fa131a5a54/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:36.750262 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ac3bf-9fc4-4126-8183-41eadcc21206 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.797876 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19763bc-8cab-4682-b44d-7a7890f8de87 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.829074 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ac3bf-9fc4-4126-8183-41eadcc21206 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.836271 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241f449-104e-4bc2-a239-002677165ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.837023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241f449-104e-4bc2-a239-002677165ab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.837306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241f449-104e-4bc2-a239-002677165ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.837710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241f449-104e-4bc2-a239-002677165ab0 admin admin] PATH_INFO: `/v3/projects/0ef53dd7ed7445d1ad4736c110b85f67/users/b81dd89ceab4437495b3beca56353d4f/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.854517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241f449-104e-4bc2-a239-002677165ab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ef53dd7ed7445d1ad4736c110b85f67, user_id=b81dd89ceab4437495b3beca56353d4f, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.857606 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241f449-104e-4bc2-a239-002677165ab0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.860293 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58596764-2cb9-418e-ad57-70c31ff5f2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.860973 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58596764-2cb9-418e-ad57-70c31ff5f2cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.861276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58596764-2cb9-418e-ad57-70c31ff5f2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.861531 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58596764-2cb9-418e-ad57-70c31ff5f2cf admin admin] PATH_INFO: `/v3/projects/2867761270a140818364c8e25f2df0e4/users/9de79389028d4ebfa448de34193f74f8/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:36.873140 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 635/1281] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:36 2022] PUT /identity/v3/projects/0ef53dd7ed7445d1ad4736c110b85f67/users/b81dd89ceab4437495b3beca56353d4f/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:36.874254 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241f449-104e-4bc2-a239-002677165ab0 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.878780 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58596764-2cb9-418e-ad57-70c31ff5f2cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2867761270a140818364c8e25f2df0e4, user_id=9de79389028d4ebfa448de34193f74f8, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:36.881532 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58596764-2cb9-418e-ad57-70c31ff5f2cf admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:36.898413 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 647/1282] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:36 2022] PUT /identity/v3/projects/2867761270a140818364c8e25f2df0e4/users/9de79389028d4ebfa448de34193f74f8/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:36.899341 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58596764-2cb9-418e-ad57-70c31ff5f2cf admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:36.947737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:36.961204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241f449-104e-4bc2-a239-002677165ab0 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.983552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58596764-2cb9-418e-ad57-70c31ff5f2cf admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:36.988734 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd1784-c0f4-4dd8-8f6f-cae9ac7ccd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:36.989406 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd1784-c0f4-4dd8-8f6f-cae9ac7ccd24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:36.989696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd1784-c0f4-4dd8-8f6f-cae9ac7ccd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:36.989957 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd1784-c0f4-4dd8-8f6f-cae9ac7ccd24 admin admin] PATH_INFO: `/v3/projects/31341466f4ef44d783aa9adae7e58362/users/fdbc39107760400baeda276ccea8825d/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:37.013123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd1784-c0f4-4dd8-8f6f-cae9ac7ccd24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31341466f4ef44d783aa9adae7e58362, user_id=fdbc39107760400baeda276ccea8825d, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:37.017493 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd1784-c0f4-4dd8-8f6f-cae9ac7ccd24 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:37.023343 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:37.041213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0b72e-b02a-49c8-92a8-9f41fec4c168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:37.043556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 648/1283] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:36 2022] PUT /identity/v3/projects/31341466f4ef44d783aa9adae7e58362/users/fdbc39107760400baeda276ccea8825d/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:37.052587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0b72e-b02a-49c8-92a8-9f41fec4c168 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:37.052587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0b72e-b02a-49c8-92a8-9f41fec4c168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:37.052587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0b72e-b02a-49c8-92a8-9f41fec4c168 admin admin] PATH_INFO: `/v3/projects/618cf0b3507845909bb27c13460d1d6c/users/1c786544e8fb4e8e8c3a4990a0f36725/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:37.053162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b0a0b-b5a2-4478-b686-651a2e3253d2 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:37.053162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b0a0b-b5a2-4478-b686-651a2e3253d2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:37.053162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b0a0b-b5a2-4478-b686-651a2e3253d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:37.068005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0b72e-b02a-49c8-92a8-9f41fec4c168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=618cf0b3507845909bb27c13460d1d6c, user_id=1c786544e8fb4e8e8c3a4990a0f36725, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:37.071554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0b72e-b02a-49c8-92a8-9f41fec4c168 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:37.072741 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-917b0a0b-b5a2-4478-b686-651a2e3253d2 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:37.074145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-917b0a0b-b5a2-4478-b686-651a2e3253d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:37.091191 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 636/1284] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:36 2022] PUT /identity/v3/projects/618cf0b3507845909bb27c13460d1d6c/users/1c786544e8fb4e8e8c3a4990a0f36725/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:37.093086 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1776aa-e7e6-4dc7-ac47-df82b9ca281e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:37.093848 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1776aa-e7e6-4dc7-ac47-df82b9ca281e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:37.095345 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1776aa-e7e6-4dc7-ac47-df82b9ca281e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:37.125278 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ae1776aa-e7e6-4dc7-ac47-df82b9ca281e None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:37.126949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ae1776aa-e7e6-4dc7-ac47-df82b9ca281e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:37.158287 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 649/1285] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:37.162999 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68685cc-db9a-4e91-b8f3-fd2bd196c91b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:37.163165 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68685cc-db9a-4e91-b8f3-fd2bd196c91b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:37.163479 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68685cc-db9a-4e91-b8f3-fd2bd196c91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:37.200766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a68685cc-db9a-4e91-b8f3-fd2bd196c91b None None] MFA Rules not processed for user `b81dd89ceab4437495b3beca56353d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:37.202000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a68685cc-db9a-4e91-b8f3-fd2bd196c91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:37.207918 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 637/1286] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:37.211666 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af0fb1-8079-4e1c-8d8a-d8c3f08b72f6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:37.211666 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af0fb1-8079-4e1c-8d8a-d8c3f08b72f6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:37.211666 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af0fb1-8079-4e1c-8d8a-d8c3f08b72f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:37.255508 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e2af0fb1-8079-4e1c-8d8a-d8c3f08b72f6 None None] MFA Rules not processed for user `9de79389028d4ebfa448de34193f74f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:37.256870 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e2af0fb1-8079-4e1c-8d8a-d8c3f08b72f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:37.290796 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 650/1287] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:37.290796 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd191a4-e1cd-4dc8-8bda-0b6206d530be None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:37.290796 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd191a4-e1cd-4dc8-8bda-0b6206d530be None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:37.290796 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd191a4-e1cd-4dc8-8bda-0b6206d530be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:37.332004 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-dbd191a4-e1cd-4dc8-8bda-0b6206d530be None None] MFA Rules not processed for user `fdbc39107760400baeda276ccea8825d`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:37.334372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dbd191a4-e1cd-4dc8-8bda-0b6206d530be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:37.366315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 638/1288] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:37.366315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e33aed-1d8f-4f2e-a298-64d0031ce4f8 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:37.366315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e33aed-1d8f-4f2e-a298-64d0031ce4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:37.366315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e33aed-1d8f-4f2e-a298-64d0031ce4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:37.396662 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-60e33aed-1d8f-4f2e-a298-64d0031ce4f8 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:37.398114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-60e33aed-1d8f-4f2e-a298-64d0031ce4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:37.405306 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:37.411462 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 639/1289] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:37.412666 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e33aed-1d8f-4f2e-a298-64d0031ce4f8 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:37.468121 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 651/1290] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:37 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:37.469201 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd191a4-e1cd-4dc8-8bda-0b6206d530be None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:37.523765 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e33aed-1d8f-4f2e-a298-64d0031ce4f8 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:37.532054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c949c18-2a52-4c62-913f-00d3fb8821e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:37.533690 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c949c18-2a52-4c62-913f-00d3fb8821e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:37.534274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c949c18-2a52-4c62-913f-00d3fb8821e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:37.534414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c949c18-2a52-4c62-913f-00d3fb8821e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:37.563417 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c949c18-2a52-4c62-913f-00d3fb8821e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:37.569575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c949c18-2a52-4c62-913f-00d3fb8821e2 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:37.575176 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd191a4-e1cd-4dc8-8bda-0b6206d530be None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:37.578963 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:37.581087 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279b58b-a9ee-44c7-b08f-efe6438cda45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:37.581976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279b58b-a9ee-44c7-b08f-efe6438cda45 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:37.582267 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279b58b-a9ee-44c7-b08f-efe6438cda45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:37.582543 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279b58b-a9ee-44c7-b08f-efe6438cda45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:37.597032 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 640/1291] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:37.597883 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279b58b-a9ee-44c7-b08f-efe6438cda45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:37.599615 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279b58b-a9ee-44c7-b08f-efe6438cda45 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:37.602672 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29474f-50b7-4a29-abe4-8498e81527bc None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:37.603190 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29474f-50b7-4a29-abe4-8498e81527bc None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:37.604176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29474f-50b7-4a29-abe4-8498e81527bc None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:37.611843 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 641/1292] 10.222.0.27 () {54 vars in 961 bytes} [Mon Oct 17 03:11:37 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:11:37.613228 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ca810-0eac-4f62-ba57-ab57a7349ae6 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:37.614082 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 652/1293] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:37.614627 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ca810-0eac-4f62-ba57-ab57a7349ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:37.614940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ca810-0eac-4f62-ba57-ab57a7349ae6 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:37.615061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279b58b-a9ee-44c7-b08f-efe6438cda45 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:37.626629 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 642/1294] 10.222.0.27 () {54 vars in 961 bytes} [Mon Oct 17 03:11:37 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:11:37.629850 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ca810-0eac-4f62-ba57-ab57a7349ae6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:37.631935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279b58b-a9ee-44c7-b08f-efe6438cda45 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:37.642938 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89054fe4-bbbb-4dea-af43-4192cdf4d96d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:37.643593 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89054fe4-bbbb-4dea-af43-4192cdf4d96d service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:37.643893 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89054fe4-bbbb-4dea-af43-4192cdf4d96d service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:37.644194 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89054fe4-bbbb-4dea-af43-4192cdf4d96d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:37.655774 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89054fe4-bbbb-4dea-af43-4192cdf4d96d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:37.657413 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89054fe4-bbbb-4dea-af43-4192cdf4d96d service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:37.658689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ca810-0eac-4f62-ba57-ab57a7349ae6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:37.668757 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 653/1295] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:37.673520 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d238257-505b-4872-a3dc-f11cbc4f0dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:37.673820 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b45295-ce7d-4318-8582-5f4809256f52 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:37.674129 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d238257-505b-4872-a3dc-f11cbc4f0dfc admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:37.674441 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b45295-ce7d-4318-8582-5f4809256f52 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:37.674573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d238257-505b-4872-a3dc-f11cbc4f0dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:37.674730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b45295-ce7d-4318-8582-5f4809256f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:37.674836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d238257-505b-4872-a3dc-f11cbc4f0dfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:37.675440 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d238257-505b-4872-a3dc-f11cbc4f0dfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:37.677415 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d238257-505b-4872-a3dc-f11cbc4f0dfc admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:37.737344 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 643/1296] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:11:37 2022] POST /identity/v3/projects => generated 405 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:37.739375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aa011-110b-417b-ba57-fa550fae49eb None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:37.739964 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aa011-110b-417b-ba57-fa550fae49eb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:37.740498 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14aa011-110b-417b-ba57-fa550fae49eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:37.761472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-13b45295-ce7d-4318-8582-5f4809256f52 None None] MFA Rules not processed for user `01637fc07ba846aebcb42325bed46216`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:37.763968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-13b45295-ce7d-4318-8582-5f4809256f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:37.774439 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e14aa011-110b-417b-ba57-fa550fae49eb None None] MFA Rules not processed for user `01637fc07ba846aebcb42325bed46216`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:37.776600 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e14aa011-110b-417b-ba57-fa550fae49eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:37.893514 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 644/1297] 10.222.0.27 () {58 vars in 1058 bytes} [Mon Oct 17 03:11:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:37.894745 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14aa011-110b-417b-ba57-fa550fae49eb None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:37.898380 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 654/1298] 10.222.0.27 () {58 vars in 1058 bytes} [Mon Oct 17 03:11:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:37.906507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b45295-ce7d-4318-8582-5f4809256f52 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:37.927297 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b45295-ce7d-4318-8582-5f4809256f52 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:37.945674 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbf2ae-e8e9-467a-9fb5-2147659b5f86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:37.946508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbf2ae-e8e9-467a-9fb5-2147659b5f86 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:37.947319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbf2ae-e8e9-467a-9fb5-2147659b5f86 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:37.947789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbf2ae-e8e9-467a-9fb5-2147659b5f86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:38.007547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14aa011-110b-417b-ba57-fa550fae49eb None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:38.013605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb42f6-5a78-465d-999c-e1bd61af4054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:38.014584 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb42f6-5a78-465d-999c-e1bd61af4054 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:38.015135 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb42f6-5a78-465d-999c-e1bd61af4054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:38.015624 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb42f6-5a78-465d-999c-e1bd61af4054 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:38.016476 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb42f6-5a78-465d-999c-e1bd61af4054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:38.018836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb42f6-5a78-465d-999c-e1bd61af4054 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:38.055087 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 645/1299] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:11:37 2022] POST /identity/v3/users => generated 309 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:38.056973 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb42f6-5a78-465d-999c-e1bd61af4054 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:38.062181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:38.088611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb42f6-5a78-465d-999c-e1bd61af4054 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:38.096698 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbf2ae-e8e9-467a-9fb5-2147659b5f86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:38.098763 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbf2ae-e8e9-467a-9fb5-2147659b5f86 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:38.108351 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fed2db-eb87-4959-b02a-19bd40c0c943 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:38.109107 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fed2db-eb87-4959-b02a-19bd40c0c943 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:38.109490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fed2db-eb87-4959-b02a-19bd40c0c943 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:38.110341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fed2db-eb87-4959-b02a-19bd40c0c943 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:38.122795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 655/1300] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:11:38.124794 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbf2ae-e8e9-467a-9fb5-2147659b5f86 service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:38.141623 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbf2ae-e8e9-467a-9fb5-2147659b5f86 service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:38.149367 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889969b5-9b68-407e-88f3-3fcd936f482e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:38.150769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889969b5-9b68-407e-88f3-3fcd936f482e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:38.151276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889969b5-9b68-407e-88f3-3fcd936f482e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:38.151755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889969b5-9b68-407e-88f3-3fcd936f482e admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:38.152658 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889969b5-9b68-407e-88f3-3fcd936f482e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:38.154771 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889969b5-9b68-407e-88f3-3fcd936f482e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:38.168786 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 656/1301] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:38.173443 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889969b5-9b68-407e-88f3-3fcd936f482e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:38.185619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889969b5-9b68-407e-88f3-3fcd936f482e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:38.192351 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95456b90-3281-4bb3-9cd0-5eb6ddabb3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:38.193313 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95456b90-3281-4bb3-9cd0-5eb6ddabb3b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:38.193804 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95456b90-3281-4bb3-9cd0-5eb6ddabb3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:38.194237 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95456b90-3281-4bb3-9cd0-5eb6ddabb3b2 admin admin] PATH_INFO: `/v3/projects/f5bc39b8903542228dbdfcb370a7be94/users/f52f1a0f003f48ec904f4fec76a994f7/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:38.217500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95456b90-3281-4bb3-9cd0-5eb6ddabb3b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5bc39b8903542228dbdfcb370a7be94, user_id=f52f1a0f003f48ec904f4fec76a994f7, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:38.220721 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95456b90-3281-4bb3-9cd0-5eb6ddabb3b2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:38.248656 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 657/1302] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:38 2022] PUT /identity/v3/projects/f5bc39b8903542228dbdfcb370a7be94/users/f52f1a0f003f48ec904f4fec76a994f7/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:38.249983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95456b90-3281-4bb3-9cd0-5eb6ddabb3b2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:38.269220 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:38.273994 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fed2db-eb87-4959-b02a-19bd40c0c943 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:38.281732 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fed2db-eb87-4959-b02a-19bd40c0c943 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:38.322052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 646/1303] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:38.420597 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95456b90-3281-4bb3-9cd0-5eb6ddabb3b2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:38.436898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7f3db-bbc6-4aae-9cb4-621d39d282d8 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:38.437541 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7f3db-bbc6-4aae-9cb4-621d39d282d8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:38.437923 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7f3db-bbc6-4aae-9cb4-621d39d282d8 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:38.439449 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 647/1304] 10.222.0.27 () {54 vars in 912 bytes} [Mon Oct 17 03:11:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:11:38.448256 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363b086-4c2d-4aac-b72b-fed9ab7d8ff8 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:38.448886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363b086-4c2d-4aac-b72b-fed9ab7d8ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:38.449274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363b086-4c2d-4aac-b72b-fed9ab7d8ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:38.450844 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1a06a-71c6-4999-a94d-161abc22b0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:38.451508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1a06a-71c6-4999-a94d-161abc22b0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:38.451796 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1a06a-71c6-4999-a94d-161abc22b0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:38.452078 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1a06a-71c6-4999-a94d-161abc22b0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:38.459156 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1a06a-71c6-4999-a94d-161abc22b0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:38.461319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1a06a-71c6-4999-a94d-161abc22b0e7 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:38.485473 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8363b086-4c2d-4aac-b72b-fed9ab7d8ff8 None None] MFA Rules not processed for user `01637fc07ba846aebcb42325bed46216`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:38.487037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8363b086-4c2d-4aac-b72b-fed9ab7d8ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:38.490599 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 658/1305] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:38.491715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1a06a-71c6-4999-a94d-161abc22b0e7 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:38.517232 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1a06a-71c6-4999-a94d-161abc22b0e7 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:38.529667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e666ed-9f9d-4b42-b039-38e2fb6094e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:38.530370 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e666ed-9f9d-4b42-b039-38e2fb6094e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:38.530714 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e666ed-9f9d-4b42-b039-38e2fb6094e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:38.532286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e666ed-9f9d-4b42-b039-38e2fb6094e2 admin admin] PATH_INFO: `/v3/projects/f5bc39b8903542228dbdfcb370a7be94/users/f52f1a0f003f48ec904f4fec76a994f7/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:38.538776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e666ed-9f9d-4b42-b039-38e2fb6094e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5bc39b8903542228dbdfcb370a7be94, user_id=f52f1a0f003f48ec904f4fec76a994f7, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:38.541525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e666ed-9f9d-4b42-b039-38e2fb6094e2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:38.581138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 659/1306] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:38 2022] PUT /identity/v3/projects/f5bc39b8903542228dbdfcb370a7be94/users/f52f1a0f003f48ec904f4fec76a994f7/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:38.582745 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74583f85-67a5-4684-beb8-25193baa2e3c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:38.583071 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74583f85-67a5-4684-beb8-25193baa2e3c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:38.583353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74583f85-67a5-4684-beb8-25193baa2e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:38.643942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 648/1307] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Oct 17 03:11:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:11:38.646105 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-74583f85-67a5-4684-beb8-25193baa2e3c None None] MFA Rules not processed for user `f52f1a0f003f48ec904f4fec76a994f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:38.647369 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-74583f85-67a5-4684-beb8-25193baa2e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:38.659709 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363b086-4c2d-4aac-b72b-fed9ab7d8ff8 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:38.734828 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363b086-4c2d-4aac-b72b-fed9ab7d8ff8 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:38.739841 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 660/1308] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:38 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:38.745567 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f4584-f98f-4a7d-b636-cdb6b3377c0a None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:38.745898 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f4584-f98f-4a7d-b636-cdb6b3377c0a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:38.746309 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f4584-f98f-4a7d-b636-cdb6b3377c0a None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:38.748452 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 661/1309] 10.222.0.27 () {54 vars in 912 bytes} [Mon Oct 17 03:11:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:11:38.756592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfae63-5549-4f55-8d59-dcc67fb17be2 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:38.757052 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfae63-5549-4f55-8d59-dcc67fb17be2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:38.758361 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfae63-5549-4f55-8d59-dcc67fb17be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:38.789007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f0cfae63-5549-4f55-8d59-dcc67fb17be2 None None] MFA Rules not processed for user `01637fc07ba846aebcb42325bed46216`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:38.790844 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f0cfae63-5549-4f55-8d59-dcc67fb17be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:38.804042 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:38.845802 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2ae5d-59b9-40d3-aed3-b9b0d1d123bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:38.846714 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2ae5d-59b9-40d3-aed3-b9b0d1d123bc service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:38.847806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2ae5d-59b9-40d3-aed3-b9b0d1d123bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:38.848266 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2ae5d-59b9-40d3-aed3-b9b0d1d123bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:38.909067 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 662/1310] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Oct 17 03:11:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:38.909942 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfae63-5549-4f55-8d59-dcc67fb17be2 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:38.921660 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfae63-5549-4f55-8d59-dcc67fb17be2 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:38.925868 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becb900-6143-49f5-a555-1f12cd905490 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:38.926521 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becb900-6143-49f5-a555-1f12cd905490 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:38.926836 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becb900-6143-49f5-a555-1f12cd905490 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:38.927227 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becb900-6143-49f5-a555-1f12cd905490 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:38.959804 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becb900-6143-49f5-a555-1f12cd905490 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:38.964633 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becb900-6143-49f5-a555-1f12cd905490 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:38.965241 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:38.990222 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 663/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:38 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:38.993634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2ae5d-59b9-40d3-aed3-b9b0d1d123bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:38.995778 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2ae5d-59b9-40d3-aed3-b9b0d1d123bc service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:39.015817 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 649/1312] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:11:39.211653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becb900-6143-49f5-a555-1f12cd905490 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:39.211653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becb900-6143-49f5-a555-1f12cd905490 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:39.211653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5fca6-03a1-4f90-af5d-6d6dc0d7f7db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:39.211653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5fca6-03a1-4f90-af5d-6d6dc0d7f7db service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:39.249629 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5fca6-03a1-4f90-af5d-6d6dc0d7f7db service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:39.249629 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5fca6-03a1-4f90-af5d-6d6dc0d7f7db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:39.249629 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5fca6-03a1-4f90-af5d-6d6dc0d7f7db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:39.249629 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5fca6-03a1-4f90-af5d-6d6dc0d7f7db service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:39.251356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 664/1313] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:40.314632 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2ae5d-59b9-40d3-aed3-b9b0d1d123bc service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:40.514292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2ae5d-59b9-40d3-aed3-b9b0d1d123bc service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:40.524802 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad22970-e228-4bd8-98f9-8379943c0d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:40.526539 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad22970-e228-4bd8-98f9-8379943c0d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:40.527090 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad22970-e228-4bd8-98f9-8379943c0d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:40.527652 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad22970-e228-4bd8-98f9-8379943c0d29 admin admin] PATH_INFO: `/v3/users/fdbc39107760400baeda276ccea8825d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:40.529383 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad22970-e228-4bd8-98f9-8379943c0d29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdbc39107760400baeda276ccea8825d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:40.532821 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad22970-e228-4bd8-98f9-8379943c0d29 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:40.655421 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2ad22970-e228-4bd8-98f9-8379943c0d29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdbc39107760400baeda276ccea8825d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:40.676507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2ad22970-e228-4bd8-98f9-8379943c0d29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdbc39107760400baeda276ccea8825d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:40.677711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2ad22970-e228-4bd8-98f9-8379943c0d29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdbc39107760400baeda276ccea8825d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:40.694467 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2ad22970-e228-4bd8-98f9-8379943c0d29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdbc39107760400baeda276ccea8825d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:40.720227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2ad22970-e228-4bd8-98f9-8379943c0d29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdbc39107760400baeda276ccea8825d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:40.730021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 650/1314] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:11:40 2022] DELETE /identity/v3/users/fdbc39107760400baeda276ccea8825d => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:40.734433 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa59067-aa08-4d91-b820-ca07f4b2ad42 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:40.735323 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa59067-aa08-4d91-b820-ca07f4b2ad42 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:40.735774 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa59067-aa08-4d91-b820-ca07f4b2ad42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:40.759556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-eaa59067-aa08-4d91-b820-ca07f4b2ad42 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:40.761203 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-eaa59067-aa08-4d91-b820-ca07f4b2ad42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:40.848064 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 665/1315] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:40.864109 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad22970-e228-4bd8-98f9-8379943c0d29 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:40.866170 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2ad22970-e228-4bd8-98f9-8379943c0d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:40.980454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad22970-e228-4bd8-98f9-8379943c0d29 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:40.986097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61135f34-fce7-46ad-95bf-1011ebb7af08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:40.987029 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61135f34-fce7-46ad-95bf-1011ebb7af08 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:40.987658 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61135f34-fce7-46ad-95bf-1011ebb7af08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:40.988099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61135f34-fce7-46ad-95bf-1011ebb7af08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:41.007031 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61135f34-fce7-46ad-95bf-1011ebb7af08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:41.009243 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61135f34-fce7-46ad-95bf-1011ebb7af08 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:41.028831 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 651/1316] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:11:43.546023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa59067-aa08-4d91-b820-ca07f4b2ad42 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:43.547437 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-eaa59067-aa08-4d91-b820-ca07f4b2ad42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:43.573331 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa59067-aa08-4d91-b820-ca07f4b2ad42 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:43.584120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a1a5f-166b-487f-b4e6-fb0393a55a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:43.584947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a1a5f-166b-487f-b4e6-fb0393a55a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:43.585416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a1a5f-166b-487f-b4e6-fb0393a55a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:43.585795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a1a5f-166b-487f-b4e6-fb0393a55a78 admin admin] PATH_INFO: `/v3/projects/31341466f4ef44d783aa9adae7e58362` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:43.589634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a1a5f-166b-487f-b4e6-fb0393a55a78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31341466f4ef44d783aa9adae7e58362)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:43.593209 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a1a5f-166b-487f-b4e6-fb0393a55a78 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:43.706823 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f55a1a5f-166b-487f-b4e6-fb0393a55a78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31341466f4ef44d783aa9adae7e58362', 'request_id': 'req-f55a1a5f-166b-487f-b4e6-fb0393a55a78'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:43.723667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f55a1a5f-166b-487f-b4e6-fb0393a55a78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31341466f4ef44d783aa9adae7e58362', 'request_id': 'req-f55a1a5f-166b-487f-b4e6-fb0393a55a78'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:43.743235 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f55a1a5f-166b-487f-b4e6-fb0393a55a78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31341466f4ef44d783aa9adae7e58362', 'request_id': 'req-f55a1a5f-166b-487f-b4e6-fb0393a55a78'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:43.770490 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f55a1a5f-166b-487f-b4e6-fb0393a55a78 admin admin] The token cache is being invalidate because project 31341466f4ef44d783aa9adae7e58362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:11:43.771111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f55a1a5f-166b-487f-b4e6-fb0393a55a78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:43.773746 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-f55a1a5f-166b-487f-b4e6-fb0393a55a78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:43.780751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 666/1317] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:11:43 2022] DELETE /identity/v3/projects/31341466f4ef44d783aa9adae7e58362 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:43.781350 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61135f34-fce7-46ad-95bf-1011ebb7af08 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:43.791968 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61135f34-fce7-46ad-95bf-1011ebb7af08 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:43.796959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:43.880315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc81d49-6667-4b8b-b807-eeca70c41761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:43.881276 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc81d49-6667-4b8b-b807-eeca70c41761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:43.881742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc81d49-6667-4b8b-b807-eeca70c41761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:43.882251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc81d49-6667-4b8b-b807-eeca70c41761 admin admin] PATH_INFO: `/v3/users/f52f1a0f003f48ec904f4fec76a994f7` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:43.884208 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc81d49-6667-4b8b-b807-eeca70c41761 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f52f1a0f003f48ec904f4fec76a994f7)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:43.886739 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc81d49-6667-4b8b-b807-eeca70c41761 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:43.964300 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2dc81d49-6667-4b8b-b807-eeca70c41761 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f52f1a0f003f48ec904f4fec76a994f7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:43.974326 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2dc81d49-6667-4b8b-b807-eeca70c41761 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f52f1a0f003f48ec904f4fec76a994f7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:43.975119 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2dc81d49-6667-4b8b-b807-eeca70c41761 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f52f1a0f003f48ec904f4fec76a994f7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:43.987381 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2dc81d49-6667-4b8b-b807-eeca70c41761 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f52f1a0f003f48ec904f4fec76a994f7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:44.009167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2dc81d49-6667-4b8b-b807-eeca70c41761 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f52f1a0f003f48ec904f4fec76a994f7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:44.020078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 652/1318] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:11:43 2022] DELETE /identity/v3/users/f52f1a0f003f48ec904f4fec76a994f7 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:45.699102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a1a5f-166b-487f-b4e6-fb0393a55a78 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:45.700541 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f55a1a5f-166b-487f-b4e6-fb0393a55a78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:45.715588 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:45.732943 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a1a5f-166b-487f-b4e6-fb0393a55a78 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:45.925435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081c19f-eb0e-40fa-8d9d-f780f2632bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:45.926743 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081c19f-eb0e-40fa-8d9d-f780f2632bbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:45.927263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081c19f-eb0e-40fa-8d9d-f780f2632bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:45.928387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081c19f-eb0e-40fa-8d9d-f780f2632bbc admin admin] PATH_INFO: `/v3/users/b81dd89ceab4437495b3beca56353d4f` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:45.931731 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081c19f-eb0e-40fa-8d9d-f780f2632bbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b81dd89ceab4437495b3beca56353d4f)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:45.933446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: WARNING py.warnings [None req-c081c19f-eb0e-40fa-8d9d-f780f2632bbc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:11:45.933446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: warnings.warn(msg) Oct 17 03:11:45.933446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: Oct 17 03:11:45.936859 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081c19f-eb0e-40fa-8d9d-f780f2632bbc admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:46.149663 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c081c19f-eb0e-40fa-8d9d-f780f2632bbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b81dd89ceab4437495b3beca56353d4f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:46.192773 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc81d49-6667-4b8b-b807-eeca70c41761 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:46.194992 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2dc81d49-6667-4b8b-b807-eeca70c41761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:46.200889 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c081c19f-eb0e-40fa-8d9d-f780f2632bbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b81dd89ceab4437495b3beca56353d4f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:46.202270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c081c19f-eb0e-40fa-8d9d-f780f2632bbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b81dd89ceab4437495b3beca56353d4f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:46.253992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c081c19f-eb0e-40fa-8d9d-f780f2632bbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b81dd89ceab4437495b3beca56353d4f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:46.270116 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc81d49-6667-4b8b-b807-eeca70c41761 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:46.311536 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52668472-f6eb-498b-b12b-07de4a4e5cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:46.312227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52668472-f6eb-498b-b12b-07de4a4e5cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:46.312513 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52668472-f6eb-498b-b12b-07de4a4e5cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:46.312797 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52668472-f6eb-498b-b12b-07de4a4e5cdc admin admin] PATH_INFO: `/v3/projects/f5bc39b8903542228dbdfcb370a7be94` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:46.322787 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52668472-f6eb-498b-b12b-07de4a4e5cdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5bc39b8903542228dbdfcb370a7be94)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:46.323977 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: WARNING py.warnings [None req-52668472-f6eb-498b-b12b-07de4a4e5cdc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:11:46.323977 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: warnings.warn(msg) Oct 17 03:11:46.323977 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: Oct 17 03:11:46.325184 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52668472-f6eb-498b-b12b-07de4a4e5cdc admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:46.354801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c081c19f-eb0e-40fa-8d9d-f780f2632bbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b81dd89ceab4437495b3beca56353d4f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:46.371003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 667/1319] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:11:45 2022] DELETE /identity/v3/users/b81dd89ceab4437495b3beca56353d4f => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:46.376499 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065794e-f35b-446f-ad15-c4e8ee4c84ee None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:46.376787 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065794e-f35b-446f-ad15-c4e8ee4c84ee None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:46.377072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065794e-f35b-446f-ad15-c4e8ee4c84ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:46.492019 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9065794e-f35b-446f-ad15-c4e8ee4c84ee None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:46.496191 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9065794e-f35b-446f-ad15-c4e8ee4c84ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:46.562563 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-52668472-f6eb-498b-b12b-07de4a4e5cdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5bc39b8903542228dbdfcb370a7be94', 'request_id': 'req-52668472-f6eb-498b-b12b-07de4a4e5cdc'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:46.594458 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-52668472-f6eb-498b-b12b-07de4a4e5cdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5bc39b8903542228dbdfcb370a7be94', 'request_id': 'req-52668472-f6eb-498b-b12b-07de4a4e5cdc'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:46.626693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-52668472-f6eb-498b-b12b-07de4a4e5cdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5bc39b8903542228dbdfcb370a7be94', 'request_id': 'req-52668472-f6eb-498b-b12b-07de4a4e5cdc'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:46.663982 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-52668472-f6eb-498b-b12b-07de4a4e5cdc admin admin] The token cache is being invalidate because project f5bc39b8903542228dbdfcb370a7be94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:11:46.664620 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-52668472-f6eb-498b-b12b-07de4a4e5cdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:46.667097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-52668472-f6eb-498b-b12b-07de4a4e5cdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:46.676051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 653/1320] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:11:46 2022] DELETE /identity/v3/projects/f5bc39b8903542228dbdfcb370a7be94 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:46.685809 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bb642-b526-43ae-94fc-32d982da32fe None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:46.686101 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bb642-b526-43ae-94fc-32d982da32fe None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:46.686380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bb642-b526-43ae-94fc-32d982da32fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:46.702325 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 668/1321] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:46.736120 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b17bb642-b526-43ae-94fc-32d982da32fe None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:46.737389 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b17bb642-b526-43ae-94fc-32d982da32fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:46.754068 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:46.758900 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065794e-f35b-446f-ad15-c4e8ee4c84ee None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:46.761071 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9065794e-f35b-446f-ad15-c4e8ee4c84ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:46.816946 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:46.893744 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065794e-f35b-446f-ad15-c4e8ee4c84ee None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:46.980003 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 654/1322] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:46.982175 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446b8d2-cf95-4a44-be75-8f17c6c90199 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:46.982475 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446b8d2-cf95-4a44-be75-8f17c6c90199 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:46.982754 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446b8d2-cf95-4a44-be75-8f17c6c90199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:47.023916 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d446b8d2-cf95-4a44-be75-8f17c6c90199 None None] MFA Rules not processed for user `37e6d36d529146299970874660e92e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:47.025635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d446b8d2-cf95-4a44-be75-8f17c6c90199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:47.029555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c806f5-e63b-4d37-b3c5-50cca3152a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:47.030201 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c806f5-e63b-4d37-b3c5-50cca3152a72 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:47.030485 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c806f5-e63b-4d37-b3c5-50cca3152a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:47.030760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c806f5-e63b-4d37-b3c5-50cca3152a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:47.059868 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c806f5-e63b-4d37-b3c5-50cca3152a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:47.061578 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c806f5-e63b-4d37-b3c5-50cca3152a72 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:47.087131 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 669/1323] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:47.088537 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a4327-260e-44e2-bbe9-c8e4ce5727cf None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:47.088832 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a4327-260e-44e2-bbe9-c8e4ce5727cf None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:47.089113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a4327-260e-44e2-bbe9-c8e4ce5727cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:47.148051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ea8a4327-260e-44e2-bbe9-c8e4ce5727cf None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:47.149595 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ea8a4327-260e-44e2-bbe9-c8e4ce5727cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:47.209420 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 670/1324] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:47.227735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a4327-260e-44e2-bbe9-c8e4ce5727cf None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:47.247605 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a4327-260e-44e2-bbe9-c8e4ce5727cf None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:47.280732 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 655/1325] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:47.293057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbe9a1-3cd9-45e1-ac64-f493a0422770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:47.293935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbe9a1-3cd9-45e1-ac64-f493a0422770 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:47.294376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbe9a1-3cd9-45e1-ac64-f493a0422770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:47.295021 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbe9a1-3cd9-45e1-ac64-f493a0422770 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:47.295840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbe9a1-3cd9-45e1-ac64-f493a0422770 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:11:47.296307 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbe9a1-3cd9-45e1-ac64-f493a0422770 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:47.300304 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446b8d2-cf95-4a44-be75-8f17c6c90199 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:47.303432 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d446b8d2-cf95-4a44-be75-8f17c6c90199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:47.306595 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbe9a1-3cd9-45e1-ac64-f493a0422770 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:47.335556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 671/1326] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:11:47.341275 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbe9a1-3cd9-45e1-ac64-f493a0422770 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:47.361252 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbe9a1-3cd9-45e1-ac64-f493a0422770 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:47.381513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df190bf3-12e7-405b-b8d1-0f1399b27c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:47.382672 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df190bf3-12e7-405b-b8d1-0f1399b27c3c admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:47.383043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df190bf3-12e7-405b-b8d1-0f1399b27c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:47.383521 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df190bf3-12e7-405b-b8d1-0f1399b27c3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:47.384207 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df190bf3-12e7-405b-b8d1-0f1399b27c3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:47.389425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df190bf3-12e7-405b-b8d1-0f1399b27c3c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:47.482097 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 672/1327] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:11:47 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:11:47.485444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df190bf3-12e7-405b-b8d1-0f1399b27c3c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:47.501196 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446b8d2-cf95-4a44-be75-8f17c6c90199 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:47.583177 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:47.646796 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d4b44-993b-4e72-8021-2e86174464ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:47.648964 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d4b44-993b-4e72-8021-2e86174464ea service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:47.649601 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d4b44-993b-4e72-8021-2e86174464ea service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:47.650586 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d4b44-993b-4e72-8021-2e86174464ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:47.660510 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df190bf3-12e7-405b-b8d1-0f1399b27c3c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:47.677431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a20063-995f-4504-84df-37e06cd0fb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:47.678071 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a20063-995f-4504-84df-37e06cd0fb79 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:47.678351 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a20063-995f-4504-84df-37e06cd0fb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:47.678627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a20063-995f-4504-84df-37e06cd0fb79 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:47.679842 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a20063-995f-4504-84df-37e06cd0fb79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:47.685778 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a20063-995f-4504-84df-37e06cd0fb79 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:47.745135 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 673/1328] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:11:47 2022] POST /identity/v3/users => generated 287 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:11:47.746276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a20063-995f-4504-84df-37e06cd0fb79 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:47.752408 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-79a20063-995f-4504-84df-37e06cd0fb79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:47.765539 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d4b44-993b-4e72-8021-2e86174464ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:47.767398 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d4b44-993b-4e72-8021-2e86174464ea service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:47.780842 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a20063-995f-4504-84df-37e06cd0fb79 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:47.789443 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 656/1329] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:47.797508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccfbfb5-c5c2-4a98-aea1-f6687c98bccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:47.798264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d4b44-993b-4e72-8021-2e86174464ea service nova] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:47.798541 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfbfb5-c5c2-4a98-aea1-f6687c98bccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:47.799019 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfbfb5-c5c2-4a98-aea1-f6687c98bccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:47.799448 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfbfb5-c5c2-4a98-aea1-f6687c98bccd admin admin] PATH_INFO: `/v3/projects/0ef53dd7ed7445d1ad4736c110b85f67` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:47.800767 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccfbfb5-c5c2-4a98-aea1-f6687c98bccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ef53dd7ed7445d1ad4736c110b85f67)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:47.802843 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccfbfb5-c5c2-4a98-aea1-f6687c98bccd admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:47.824878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d4b44-993b-4e72-8021-2e86174464ea service nova] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:47.832923 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a425da-d330-4115-bb89-6a4e3d8e3a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:47.833704 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a425da-d330-4115-bb89-6a4e3d8e3a9b admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:47.834453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a425da-d330-4115-bb89-6a4e3d8e3a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:47.840355 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a425da-d330-4115-bb89-6a4e3d8e3a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:47.841241 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a425da-d330-4115-bb89-6a4e3d8e3a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:47.843214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a425da-d330-4115-bb89-6a4e3d8e3a9b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:47.870689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 657/1330] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:47.873939 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:47.880741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a425da-d330-4115-bb89-6a4e3d8e3a9b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:47.902414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a425da-d330-4115-bb89-6a4e3d8e3a9b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:47.918245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dccfbfb5-c5c2-4a98-aea1-f6687c98bccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ef53dd7ed7445d1ad4736c110b85f67', 'request_id': 'req-dccfbfb5-c5c2-4a98-aea1-f6687c98bccd'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:47.929377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dccfbfb5-c5c2-4a98-aea1-f6687c98bccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ef53dd7ed7445d1ad4736c110b85f67', 'request_id': 'req-dccfbfb5-c5c2-4a98-aea1-f6687c98bccd'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:47.942026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dccfbfb5-c5c2-4a98-aea1-f6687c98bccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ef53dd7ed7445d1ad4736c110b85f67', 'request_id': 'req-dccfbfb5-c5c2-4a98-aea1-f6687c98bccd'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:47.967508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dccfbfb5-c5c2-4a98-aea1-f6687c98bccd admin admin] The token cache is being invalidate because project 0ef53dd7ed7445d1ad4736c110b85f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:11:47.967818 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dccfbfb5-c5c2-4a98-aea1-f6687c98bccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:47.969389 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dccfbfb5-c5c2-4a98-aea1-f6687c98bccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:47.980277 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 674/1331] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:11:47 2022] DELETE /identity/v3/projects/0ef53dd7ed7445d1ad4736c110b85f67 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:47.985440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268be58-0695-4ce4-b47b-c80883ad7b7f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:47.986025 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268be58-0695-4ce4-b47b-c80883ad7b7f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:47.986512 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268be58-0695-4ce4-b47b-c80883ad7b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:48.036300 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e268be58-0695-4ce4-b47b-c80883ad7b7f None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:48.037734 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e268be58-0695-4ce4-b47b-c80883ad7b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:48.066214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc009d-5ef6-4cd3-89e1-96540ca043d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:48.067113 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc009d-5ef6-4cd3-89e1-96540ca043d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:48.072381 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc009d-5ef6-4cd3-89e1-96540ca043d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:48.072580 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc009d-5ef6-4cd3-89e1-96540ca043d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:48.082849 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc009d-5ef6-4cd3-89e1-96540ca043d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:48.082849 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc009d-5ef6-4cd3-89e1-96540ca043d2 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:48.096308 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 658/1332] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:48.108872 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc009d-5ef6-4cd3-89e1-96540ca043d2 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:48.111796 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4efc009d-5ef6-4cd3-89e1-96540ca043d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:48.154805 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc009d-5ef6-4cd3-89e1-96540ca043d2 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:48.166835 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb68e98-8a71-4fee-8e6d-33100e1bdb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:48.169421 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb68e98-8a71-4fee-8e6d-33100e1bdb80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:48.169729 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb68e98-8a71-4fee-8e6d-33100e1bdb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:48.169986 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb68e98-8a71-4fee-8e6d-33100e1bdb80 admin admin] PATH_INFO: `/v3/projects/1c26c9ea1fc64309b38b2d3345140d04/users/7c1adc3c270044cba48818007f050cde/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:48.180533 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 675/1333] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:48.189187 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de003fcc-f659-42a5-bb08-c681a8f6a231 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:48.191736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de003fcc-f659-42a5-bb08-c681a8f6a231 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:48.191736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de003fcc-f659-42a5-bb08-c681a8f6a231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:48.211439 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb68e98-8a71-4fee-8e6d-33100e1bdb80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c26c9ea1fc64309b38b2d3345140d04, user_id=7c1adc3c270044cba48818007f050cde, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:48.217994 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb68e98-8a71-4fee-8e6d-33100e1bdb80 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:48.232298 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-de003fcc-f659-42a5-bb08-c681a8f6a231 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:48.233759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-de003fcc-f659-42a5-bb08-c681a8f6a231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:48.256658 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 659/1334] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:48 2022] PUT /identity/v3/projects/1c26c9ea1fc64309b38b2d3345140d04/users/7c1adc3c270044cba48818007f050cde/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:48.256890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 676/1335] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:48.261425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fdcab-7101-419e-9231-0b3b6e16f56c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:48.261863 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fdcab-7101-419e-9231-0b3b6e16f56c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:48.262323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4fdcab-7101-419e-9231-0b3b6e16f56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:48.271449 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de003fcc-f659-42a5-bb08-c681a8f6a231 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:48.310172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de003fcc-f659-42a5-bb08-c681a8f6a231 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:48.323805 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1b4fdcab-7101-419e-9231-0b3b6e16f56c None None] MFA Rules not processed for user `7c1adc3c270044cba48818007f050cde`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:48.325214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1b4fdcab-7101-419e-9231-0b3b6e16f56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:48.376181 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:48.470552 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 660/1336] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:48.484677 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4fdcab-7101-419e-9231-0b3b6e16f56c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:48.542546 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef0238-6688-49db-a502-510bd59dfb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:48.543241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef0238-6688-49db-a502-510bd59dfb2f admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:48.543809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef0238-6688-49db-a502-510bd59dfb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:48.544129 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef0238-6688-49db-a502-510bd59dfb2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:48.544836 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef0238-6688-49db-a502-510bd59dfb2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:11:48.545405 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef0238-6688-49db-a502-510bd59dfb2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:48.547162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef0238-6688-49db-a502-510bd59dfb2f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:48.564760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 677/1337] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:11:48.565786 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef0238-6688-49db-a502-510bd59dfb2f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:48.568849 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b1ef0238-6688-49db-a502-510bd59dfb2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:48.600586 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef0238-6688-49db-a502-510bd59dfb2f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:48.602665 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4fdcab-7101-419e-9231-0b3b6e16f56c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:48.608209 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd14b29-c69f-458f-a389-91a6c6b583ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:48.616228 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2447cd-23b1-481b-8f34-0bff1fc36905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:48.619348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd14b29-c69f-458f-a389-91a6c6b583ad admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:48.619850 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd14b29-c69f-458f-a389-91a6c6b583ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:48.620242 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd14b29-c69f-458f-a389-91a6c6b583ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:48.621103 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd14b29-c69f-458f-a389-91a6c6b583ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:48.623554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2447cd-23b1-481b-8f34-0bff1fc36905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:48.624024 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2447cd-23b1-481b-8f34-0bff1fc36905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:48.624456 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2447cd-23b1-481b-8f34-0bff1fc36905 admin admin] PATH_INFO: `/v3/users/9de79389028d4ebfa448de34193f74f8` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:48.626059 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2447cd-23b1-481b-8f34-0bff1fc36905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9de79389028d4ebfa448de34193f74f8)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:48.627249 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd14b29-c69f-458f-a389-91a6c6b583ad admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:48.629461 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2447cd-23b1-481b-8f34-0bff1fc36905 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:48.678013 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 661/1338] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:11:48 2022] POST /identity/v3/projects => generated 373 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:48.679264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd14b29-c69f-458f-a389-91a6c6b583ad admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:48.738513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:48.765484 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9b2447cd-23b1-481b-8f34-0bff1fc36905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9de79389028d4ebfa448de34193f74f8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:48.785282 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9b2447cd-23b1-481b-8f34-0bff1fc36905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9de79389028d4ebfa448de34193f74f8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:48.786177 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9b2447cd-23b1-481b-8f34-0bff1fc36905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9de79389028d4ebfa448de34193f74f8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:48.804684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9b2447cd-23b1-481b-8f34-0bff1fc36905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9de79389028d4ebfa448de34193f74f8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:48.832220 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9b2447cd-23b1-481b-8f34-0bff1fc36905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9de79389028d4ebfa448de34193f74f8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:48.839690 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd14b29-c69f-458f-a389-91a6c6b583ad admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:48.845808 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 678/1339] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:11:48 2022] DELETE /identity/v3/users/9de79389028d4ebfa448de34193f74f8 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:48.850359 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2447cd-23b1-481b-8f34-0bff1fc36905 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:48.857236 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9b2447cd-23b1-481b-8f34-0bff1fc36905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:48.858452 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a5f30-946a-4f5d-bf4a-1f9e9d01feba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:48.859207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a5f30-946a-4f5d-bf4a-1f9e9d01feba admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:48.859521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a5f30-946a-4f5d-bf4a-1f9e9d01feba admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:48.859800 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a5f30-946a-4f5d-bf4a-1f9e9d01feba admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:48.860452 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a5f30-946a-4f5d-bf4a-1f9e9d01feba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:48.862427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a5f30-946a-4f5d-bf4a-1f9e9d01feba admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:48.932376 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 662/1340] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:11:48 2022] POST /identity/v3/projects => generated 369 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:48.936113 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f7d87-a9d1-4fe4-afb6-44fb71b4ac05 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:48.941226 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f7d87-a9d1-4fe4-afb6-44fb71b4ac05 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:48.944744 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f7d87-a9d1-4fe4-afb6-44fb71b4ac05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:48.968061 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5c8f7d87-a9d1-4fe4-afb6-44fb71b4ac05 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:48.969991 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5c8f7d87-a9d1-4fe4-afb6-44fb71b4ac05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:49.017802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2447cd-23b1-481b-8f34-0bff1fc36905 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:49.059809 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:49.149533 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 663/1341] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:49.150779 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f7d87-a9d1-4fe4-afb6-44fb71b4ac05 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:49.153406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5c8f7d87-a9d1-4fe4-afb6-44fb71b4ac05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:49.180312 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f7d87-a9d1-4fe4-afb6-44fb71b4ac05 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:49.181080 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f78ad6-0b53-4f7b-a1bf-07f668b37749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:49.181892 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f78ad6-0b53-4f7b-a1bf-07f668b37749 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:49.183128 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f78ad6-0b53-4f7b-a1bf-07f668b37749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:49.183563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f78ad6-0b53-4f7b-a1bf-07f668b37749 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:49.184396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f78ad6-0b53-4f7b-a1bf-07f668b37749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:49.186789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f78ad6-0b53-4f7b-a1bf-07f668b37749 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:49.193656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0c541-7bc7-4ea2-beae-0aa0cb1bd8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:49.194292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0c541-7bc7-4ea2-beae-0aa0cb1bd8ea admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:49.195305 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0c541-7bc7-4ea2-beae-0aa0cb1bd8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:49.195732 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0c541-7bc7-4ea2-beae-0aa0cb1bd8ea admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:49.196601 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0c541-7bc7-4ea2-beae-0aa0cb1bd8ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:49.198589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0c541-7bc7-4ea2-beae-0aa0cb1bd8ea admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:49.234989 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 679/1342] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:11:48 2022] POST /identity/v3/users => generated 293 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:49.236281 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f78ad6-0b53-4f7b-a1bf-07f668b37749 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:49.238527 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f3f78ad6-0b53-4f7b-a1bf-07f668b37749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:49.264509 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 664/1343] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:11:49 2022] POST /identity/v3/users => generated 285 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:49.265624 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0c541-7bc7-4ea2-beae-0aa0cb1bd8ea admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:49.298205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0c541-7bc7-4ea2-beae-0aa0cb1bd8ea admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:49.310420 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456d51f-7d05-42f3-92c4-061fa8dcdd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:49.311352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456d51f-7d05-42f3-92c4-061fa8dcdd89 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:49.312477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456d51f-7d05-42f3-92c4-061fa8dcdd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:49.312942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456d51f-7d05-42f3-92c4-061fa8dcdd89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:49.313963 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456d51f-7d05-42f3-92c4-061fa8dcdd89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:49.316751 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456d51f-7d05-42f3-92c4-061fa8dcdd89 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:49.334544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 665/1344] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:49.336629 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456d51f-7d05-42f3-92c4-061fa8dcdd89 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:49.360094 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456d51f-7d05-42f3-92c4-061fa8dcdd89 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:49.368852 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec85e05-9e71-47eb-a9d4-62c9d6d317c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:49.370052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec85e05-9e71-47eb-a9d4-62c9d6d317c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:49.370512 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec85e05-9e71-47eb-a9d4-62c9d6d317c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:49.370969 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec85e05-9e71-47eb-a9d4-62c9d6d317c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:49.371794 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec85e05-9e71-47eb-a9d4-62c9d6d317c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:49.374442 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec85e05-9e71-47eb-a9d4-62c9d6d317c4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:49.394082 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 666/1345] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:49.395473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:49.397020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec85e05-9e71-47eb-a9d4-62c9d6d317c4 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:49.401954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f78ad6-0b53-4f7b-a1bf-07f668b37749 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:49.415467 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec85e05-9e71-47eb-a9d4-62c9d6d317c4 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:49.415964 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ad82e-9fbe-44bb-9479-9c4ea7a61d00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:49.416611 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ad82e-9fbe-44bb-9479-9c4ea7a61d00 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:49.416895 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ad82e-9fbe-44bb-9479-9c4ea7a61d00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:49.417171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ad82e-9fbe-44bb-9479-9c4ea7a61d00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:49.422294 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b11f2-0f46-4cc1-a0b1-f976752f1152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:49.423272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b11f2-0f46-4cc1-a0b1-f976752f1152 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:49.423705 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b11f2-0f46-4cc1-a0b1-f976752f1152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:49.424136 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b11f2-0f46-4cc1-a0b1-f976752f1152 admin admin] PATH_INFO: `/v3/projects/8e4b58b189e2470dabf95cbdce14bd08/users/3f34e58670c449c19718caba423ac5d7/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:49.435301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ad82e-9fbe-44bb-9479-9c4ea7a61d00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:49.437200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ad82e-9fbe-44bb-9479-9c4ea7a61d00 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:49.458882 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 680/1346] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:49.460018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ad82e-9fbe-44bb-9479-9c4ea7a61d00 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:49.473235 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b11f2-0f46-4cc1-a0b1-f976752f1152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e4b58b189e2470dabf95cbdce14bd08, user_id=3f34e58670c449c19718caba423ac5d7, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:49.478810 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ad82e-9fbe-44bb-9479-9c4ea7a61d00 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:49.481753 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b11f2-0f46-4cc1-a0b1-f976752f1152 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:49.485266 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c7ec9-b32e-473e-b994-ad5d40d3d18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:49.486082 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c7ec9-b32e-473e-b994-ad5d40d3d18e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:49.486559 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c7ec9-b32e-473e-b994-ad5d40d3d18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:49.487040 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c7ec9-b32e-473e-b994-ad5d40d3d18e admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:49.488142 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c7ec9-b32e-473e-b994-ad5d40d3d18e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:49.491710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c7ec9-b32e-473e-b994-ad5d40d3d18e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:49.509940 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 681/1347] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:49.516276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c7ec9-b32e-473e-b994-ad5d40d3d18e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:49.529680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 667/1348] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:49 2022] PUT /identity/v3/projects/8e4b58b189e2470dabf95cbdce14bd08/users/3f34e58670c449c19718caba423ac5d7/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:49.533795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b11f2-0f46-4cc1-a0b1-f976752f1152 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:49.536648 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c7ec9-b32e-473e-b994-ad5d40d3d18e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:49.645167 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54ff08-7f05-479d-95ef-33a99c494b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:49.646432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54ff08-7f05-479d-95ef-33a99c494b47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:49.646952 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54ff08-7f05-479d-95ef-33a99c494b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:49.647402 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54ff08-7f05-479d-95ef-33a99c494b47 admin admin] PATH_INFO: `/v3/projects/3ee67a643720454c942d616c09ab2f0f/users/7655988a5c374922b937617681e3798d/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:49.673482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54ff08-7f05-479d-95ef-33a99c494b47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee67a643720454c942d616c09ab2f0f, user_id=7655988a5c374922b937617681e3798d, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:49.677673 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54ff08-7f05-479d-95ef-33a99c494b47 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:49.697486 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 682/1349] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:49 2022] PUT /identity/v3/projects/3ee67a643720454c942d616c09ab2f0f/users/7655988a5c374922b937617681e3798d/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:49.708968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fe070-66a6-4979-9015-a11e035ce239 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:49.709862 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fe070-66a6-4979-9015-a11e035ce239 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:49.709862 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fe070-66a6-4979-9015-a11e035ce239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:49.763395 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b11f2-0f46-4cc1-a0b1-f976752f1152 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:49.775291 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9d8fe070-66a6-4979-9015-a11e035ce239 None None] MFA Rules not processed for user `7655988a5c374922b937617681e3798d`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:49.779240 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9d8fe070-66a6-4979-9015-a11e035ce239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:49.900228 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409ffa9-f11d-44e9-8250-5f951d474028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:49.901023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409ffa9-f11d-44e9-8250-5f951d474028 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:49.901303 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409ffa9-f11d-44e9-8250-5f951d474028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:49.901573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409ffa9-f11d-44e9-8250-5f951d474028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:49.902224 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409ffa9-f11d-44e9-8250-5f951d474028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:49.905428 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409ffa9-f11d-44e9-8250-5f951d474028 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:49.945194 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 683/1351] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:49.946688 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 668/1351] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:49.948671 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fe070-66a6-4979-9015-a11e035ce239 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:49.949741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409ffa9-f11d-44e9-8250-5f951d474028 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:49.977735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fe070-66a6-4979-9015-a11e035ce239 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:49.978912 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409ffa9-f11d-44e9-8250-5f951d474028 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:49.989424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cbb65-34b0-4146-9b6b-3d9a0557f53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:49.992170 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f264be1-ed1b-474b-8eb6-646cb6bb3b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:49.992868 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f264be1-ed1b-474b-8eb6-646cb6bb3b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:49.993084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cbb65-34b0-4146-9b6b-3d9a0557f53a admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:49.993211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f264be1-ed1b-474b-8eb6-646cb6bb3b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:49.993401 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f264be1-ed1b-474b-8eb6-646cb6bb3b00 admin admin] PATH_INFO: `/v3/projects/8e4b58b189e2470dabf95cbdce14bd08/users/3f34e58670c449c19718caba423ac5d7/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:49.993754 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cbb65-34b0-4146-9b6b-3d9a0557f53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:49.995077 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cbb65-34b0-4146-9b6b-3d9a0557f53a admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:49.996052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3cbb65-34b0-4146-9b6b-3d9a0557f53a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:49.996187 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f264be1-ed1b-474b-8eb6-646cb6bb3b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e4b58b189e2470dabf95cbdce14bd08, user_id=3f34e58670c449c19718caba423ac5d7, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:49.999672 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f264be1-ed1b-474b-8eb6-646cb6bb3b00 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:50.000653 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3cbb65-34b0-4146-9b6b-3d9a0557f53a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:50.016715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 684/1352] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:49 2022] PUT /identity/v3/projects/8e4b58b189e2470dabf95cbdce14bd08/users/3f34e58670c449c19718caba423ac5d7/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:50.028935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39ac85-77b2-4e35-afc5-d6d16c096aad None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:50.029231 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39ac85-77b2-4e35-afc5-d6d16c096aad None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:50.032199 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39ac85-77b2-4e35-afc5-d6d16c096aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:50.039490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:50.064718 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 669/1353] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:11:49 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:50.066118 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cbb65-34b0-4146-9b6b-3d9a0557f53a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:50.103751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ae39ac85-77b2-4e35-afc5-d6d16c096aad None None] MFA Rules not processed for user `3f34e58670c449c19718caba423ac5d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:50.105150 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ae39ac85-77b2-4e35-afc5-d6d16c096aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:50.166385 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3cbb65-34b0-4146-9b6b-3d9a0557f53a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:50.172155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:50.174892 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de02a6-d0da-4987-a5d6-d798ce1f3b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:50.175907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de02a6-d0da-4987-a5d6-d798ce1f3b21 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:50.176259 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de02a6-d0da-4987-a5d6-d798ce1f3b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:50.176815 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de02a6-d0da-4987-a5d6-d798ce1f3b21 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:50.178786 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de02a6-d0da-4987-a5d6-d798ce1f3b21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:50.183796 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de02a6-d0da-4987-a5d6-d798ce1f3b21 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:50.224940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 670/1354] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:11:50 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:11:50.228601 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de02a6-d0da-4987-a5d6-d798ce1f3b21 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:50.252248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de02a6-d0da-4987-a5d6-d798ce1f3b21 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:50.260400 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120ad39-64e1-44cf-88a9-8c522776b36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:50.261034 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120ad39-64e1-44cf-88a9-8c522776b36a admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:50.261362 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120ad39-64e1-44cf-88a9-8c522776b36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:50.262170 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120ad39-64e1-44cf-88a9-8c522776b36a admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:50.262901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120ad39-64e1-44cf-88a9-8c522776b36a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:50.266466 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120ad39-64e1-44cf-88a9-8c522776b36a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:50.283524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 685/1355] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:50.300162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39ac85-77b2-4e35-afc5-d6d16c096aad None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:50.301971 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ae39ac85-77b2-4e35-afc5-d6d16c096aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:50.307216 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 671/1356] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:50.312158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120ad39-64e1-44cf-88a9-8c522776b36a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:50.335880 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120ad39-64e1-44cf-88a9-8c522776b36a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:50.348083 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3a975-0466-4c13-8a8f-d16f04460d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:50.349245 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3a975-0466-4c13-8a8f-d16f04460d32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:50.350528 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3a975-0466-4c13-8a8f-d16f04460d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:50.351952 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3a975-0466-4c13-8a8f-d16f04460d32 admin admin] PATH_INFO: `/v3/projects/01cff7d6b3784769931015eba05331b5/users/5134908908d444adb9994733f2620f7e/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:50.392058 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3a975-0466-4c13-8a8f-d16f04460d32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01cff7d6b3784769931015eba05331b5, user_id=5134908908d444adb9994733f2620f7e, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:50.394990 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3a975-0466-4c13-8a8f-d16f04460d32 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:50.403762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39ac85-77b2-4e35-afc5-d6d16c096aad None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:50.409145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975600e-ac5a-46b9-b8af-8b92f5c4c624 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:50.410762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975600e-ac5a-46b9-b8af-8b92f5c4c624 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:50.411509 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975600e-ac5a-46b9-b8af-8b92f5c4c624 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:50.411982 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975600e-ac5a-46b9-b8af-8b92f5c4c624 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:50.418184 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 672/1357] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:50 2022] PUT /identity/v3/projects/01cff7d6b3784769931015eba05331b5/users/5134908908d444adb9994733f2620f7e/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:11:50.421999 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3a975-0466-4c13-8a8f-d16f04460d32 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:50.430367 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975600e-ac5a-46b9-b8af-8b92f5c4c624 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:50.433080 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975600e-ac5a-46b9-b8af-8b92f5c4c624 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:50.461060 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 686/1358] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:50.586944 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3a975-0466-4c13-8a8f-d16f04460d32 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:50.603435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1a1da-8deb-4b18-b938-eda25ac25f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:50.605446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1a1da-8deb-4b18-b938-eda25ac25f93 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:50.605714 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1a1da-8deb-4b18-b938-eda25ac25f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:50.605994 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1a1da-8deb-4b18-b938-eda25ac25f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:50.606656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1a1da-8deb-4b18-b938-eda25ac25f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:50.608311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1a1da-8deb-4b18-b938-eda25ac25f93 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:50.648184 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 673/1359] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:11:50.653643 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975600e-ac5a-46b9-b8af-8b92f5c4c624 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:50.684920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975600e-ac5a-46b9-b8af-8b92f5c4c624 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:50.698073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ecdd6-34ac-4956-999b-914084d53333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:50.699200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ecdd6-34ac-4956-999b-914084d53333 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:50.700382 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ecdd6-34ac-4956-999b-914084d53333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:50.700982 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ecdd6-34ac-4956-999b-914084d53333 admin admin] PATH_INFO: `/v3/projects/01cff7d6b3784769931015eba05331b5/users/5134908908d444adb9994733f2620f7e/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:50.707169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ecdd6-34ac-4956-999b-914084d53333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01cff7d6b3784769931015eba05331b5, user_id=5134908908d444adb9994733f2620f7e, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:50.711134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ecdd6-34ac-4956-999b-914084d53333 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:50.744930 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 687/1360] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:50 2022] PUT /identity/v3/projects/01cff7d6b3784769931015eba05331b5/users/5134908908d444adb9994733f2620f7e/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:50.755298 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b888282-0667-44bf-948a-e471b913e374 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:50.755490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b888282-0667-44bf-948a-e471b913e374 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:50.755734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b888282-0667-44bf-948a-e471b913e374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:50.823968 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-6b888282-0667-44bf-948a-e471b913e374 None None] MFA Rules not processed for user `5134908908d444adb9994733f2620f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:50.825402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6b888282-0667-44bf-948a-e471b913e374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:50.908873 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:51.003308 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 674/1361] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:51.021723 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ecdd6-34ac-4956-999b-914084d53333 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:51.196124 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ecdd6-34ac-4956-999b-914084d53333 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:51.211441 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ab42b-3c38-4290-a349-71cda271208f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:51.213033 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ab42b-3c38-4290-a349-71cda271208f service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:51.213538 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ab42b-3c38-4290-a349-71cda271208f service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:51.217594 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ab42b-3c38-4290-a349-71cda271208f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:51.250658 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ab42b-3c38-4290-a349-71cda271208f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:51.257263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ab42b-3c38-4290-a349-71cda271208f service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:51.296186 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 688/1362] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:51.449107 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b888282-0667-44bf-948a-e471b913e374 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:51.453803 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ab42b-3c38-4290-a349-71cda271208f service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:51.468052 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ab42b-3c38-4290-a349-71cda271208f service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:51.474360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb5193-22b8-487d-9e52-0e793246bd4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:51.477454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb5193-22b8-487d-9e52-0e793246bd4d service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:51.478014 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb5193-22b8-487d-9e52-0e793246bd4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:51.478433 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb5193-22b8-487d-9e52-0e793246bd4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:51.563073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b888282-0667-44bf-948a-e471b913e374 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:51.572716 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e47a0d-2e6d-46b1-887f-1cebbcbd4691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:51.573282 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e47a0d-2e6d-46b1-887f-1cebbcbd4691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:51.573563 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e47a0d-2e6d-46b1-887f-1cebbcbd4691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:51.573838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e47a0d-2e6d-46b1-887f-1cebbcbd4691 admin admin] PATH_INFO: `/v3/users/7c1adc3c270044cba48818007f050cde` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:51.575812 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e47a0d-2e6d-46b1-887f-1cebbcbd4691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c1adc3c270044cba48818007f050cde)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:51.586047 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e47a0d-2e6d-46b1-887f-1cebbcbd4691 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:51.668963 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:51.675666 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a8e47a0d-2e6d-46b1-887f-1cebbcbd4691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c1adc3c270044cba48818007f050cde'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:51.678820 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb5193-22b8-487d-9e52-0e793246bd4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:51.685467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb5193-22b8-487d-9e52-0e793246bd4d service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:51.698705 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a8e47a0d-2e6d-46b1-887f-1cebbcbd4691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c1adc3c270044cba48818007f050cde'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:51.699500 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a8e47a0d-2e6d-46b1-887f-1cebbcbd4691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c1adc3c270044cba48818007f050cde'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:51.710402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a8e47a0d-2e6d-46b1-887f-1cebbcbd4691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c1adc3c270044cba48818007f050cde'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:51.731520 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a8e47a0d-2e6d-46b1-887f-1cebbcbd4691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c1adc3c270044cba48818007f050cde'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:51.738238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 689/1363] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:11:51.741585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb5193-22b8-487d-9e52-0e793246bd4d service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:51.749933 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 675/1364] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:11:51 2022] DELETE /identity/v3/users/7c1adc3c270044cba48818007f050cde => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:51.753521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ede44-c677-4fe9-a652-fbe33f98ba51 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:51.753817 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ede44-c677-4fe9-a652-fbe33f98ba51 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:51.754200 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ede44-c677-4fe9-a652-fbe33f98ba51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:51.761365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-66eb5193-22b8-487d-9e52-0e793246bd4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:51.803477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-628ede44-c677-4fe9-a652-fbe33f98ba51 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:51.804834 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-628ede44-c677-4fe9-a652-fbe33f98ba51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:51.832265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb5193-22b8-487d-9e52-0e793246bd4d service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:51.929614 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:51.973641 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 676/1365] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:52.003220 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ede44-c677-4fe9-a652-fbe33f98ba51 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:52.004543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-628ede44-c677-4fe9-a652-fbe33f98ba51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:52.106569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2c994-39c5-4c93-9202-54cf349baa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:52.107420 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2c994-39c5-4c93-9202-54cf349baa88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:52.107811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2c994-39c5-4c93-9202-54cf349baa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:52.108198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2c994-39c5-4c93-9202-54cf349baa88 admin admin] PATH_INFO: `/v3/projects/2867761270a140818364c8e25f2df0e4` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:52.110129 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2c994-39c5-4c93-9202-54cf349baa88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2867761270a140818364c8e25f2df0e4)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:52.112233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2c994-39c5-4c93-9202-54cf349baa88 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:52.139248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ede44-c677-4fe9-a652-fbe33f98ba51 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:52.143224 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba19ce-504f-46c8-87f6-eaefd636f5af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:52.150027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba19ce-504f-46c8-87f6-eaefd636f5af service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:52.150739 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba19ce-504f-46c8-87f6-eaefd636f5af service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:52.151071 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba19ce-504f-46c8-87f6-eaefd636f5af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:52.162629 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba19ce-504f-46c8-87f6-eaefd636f5af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:52.164260 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba19ce-504f-46c8-87f6-eaefd636f5af service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:52.177320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 677/1366] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:11:52.320819 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dde2c994-39c5-4c93-9202-54cf349baa88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2867761270a140818364c8e25f2df0e4', 'request_id': 'req-dde2c994-39c5-4c93-9202-54cf349baa88'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:52.338404 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dde2c994-39c5-4c93-9202-54cf349baa88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2867761270a140818364c8e25f2df0e4', 'request_id': 'req-dde2c994-39c5-4c93-9202-54cf349baa88'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:52.349085 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dde2c994-39c5-4c93-9202-54cf349baa88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2867761270a140818364c8e25f2df0e4', 'request_id': 'req-dde2c994-39c5-4c93-9202-54cf349baa88'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:52.374265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dde2c994-39c5-4c93-9202-54cf349baa88 admin admin] The token cache is being invalidate because project 2867761270a140818364c8e25f2df0e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:11:52.374727 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dde2c994-39c5-4c93-9202-54cf349baa88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:52.376747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dde2c994-39c5-4c93-9202-54cf349baa88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:52.384040 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 690/1367] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:11:51 2022] DELETE /identity/v3/projects/2867761270a140818364c8e25f2df0e4 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:52.385968 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a461b-35d2-4e27-a2ff-13c88bf34e09 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:52.386256 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a461b-35d2-4e27-a2ff-13c88bf34e09 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:52.386531 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a461b-35d2-4e27-a2ff-13c88bf34e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:52.414730 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2e4a461b-35d2-4e27-a2ff-13c88bf34e09 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:52.416006 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2e4a461b-35d2-4e27-a2ff-13c88bf34e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:52.547460 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 678/1368] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:52.548514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c485d0-51e7-47a4-8bba-cb6f003535c3 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:52.548806 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c485d0-51e7-47a4-8bba-cb6f003535c3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:52.549083 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c485d0-51e7-47a4-8bba-cb6f003535c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:52.586564 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-49c485d0-51e7-47a4-8bba-cb6f003535c3 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:52.591158 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-49c485d0-51e7-47a4-8bba-cb6f003535c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:52.609153 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 691/1369] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:11:52.614376 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a461b-35d2-4e27-a2ff-13c88bf34e09 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:52.630175 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a461b-35d2-4e27-a2ff-13c88bf34e09 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:52.636718 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9ab75-1e27-4fc2-8864-dd9fc7e1fe6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:52.637530 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9ab75-1e27-4fc2-8864-dd9fc7e1fe6b admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:52.637919 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9ab75-1e27-4fc2-8864-dd9fc7e1fe6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:52.638301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9ab75-1e27-4fc2-8864-dd9fc7e1fe6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:52.639411 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f9ab75-1e27-4fc2-8864-dd9fc7e1fe6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:11:52.639842 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f9ab75-1e27-4fc2-8864-dd9fc7e1fe6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:52.641712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f9ab75-1e27-4fc2-8864-dd9fc7e1fe6b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:52.677096 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 679/1370] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:11:52.688162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c485d0-51e7-47a4-8bba-cb6f003535c3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:52.711732 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c485d0-51e7-47a4-8bba-cb6f003535c3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:52.717962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96be689-a500-4b0c-acfc-603587e7407c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:52.718984 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96be689-a500-4b0c-acfc-603587e7407c admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:52.719433 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96be689-a500-4b0c-acfc-603587e7407c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:52.719820 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96be689-a500-4b0c-acfc-603587e7407c admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:52.720586 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96be689-a500-4b0c-acfc-603587e7407c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:52.722907 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96be689-a500-4b0c-acfc-603587e7407c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:52.795177 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 692/1371] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:11:52 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:52.795389 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9ab75-1e27-4fc2-8864-dd9fc7e1fe6b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:52.812545 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9ab75-1e27-4fc2-8864-dd9fc7e1fe6b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:52.971548 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7d509-3e3f-4219-885e-cb4e1ba489f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:52.972191 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7d509-3e3f-4219-885e-cb4e1ba489f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:52.972471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7d509-3e3f-4219-885e-cb4e1ba489f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:52.972742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7d509-3e3f-4219-885e-cb4e1ba489f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:52.973408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7d509-3e3f-4219-885e-cb4e1ba489f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:52.978793 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7d509-3e3f-4219-885e-cb4e1ba489f3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:53.037954 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 680/1372] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:11:52 2022] POST /identity/v3/users => generated 283 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:11:53.042722 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96be689-a500-4b0c-acfc-603587e7407c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:53.057742 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96be689-a500-4b0c-acfc-603587e7407c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:53.071317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e4af5-f70e-48f6-ade0-a389aad21d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:53.075619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e4af5-f70e-48f6-ade0-a389aad21d08 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:53.075947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e4af5-f70e-48f6-ade0-a389aad21d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:53.076229 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e4af5-f70e-48f6-ade0-a389aad21d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:53.078331 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e4af5-f70e-48f6-ade0-a389aad21d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:53.083589 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e4af5-f70e-48f6-ade0-a389aad21d08 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:53.118711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7d509-3e3f-4219-885e-cb4e1ba489f3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:53.134992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 693/1373] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:53.136419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7d509-3e3f-4219-885e-cb4e1ba489f3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:53.147339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d00c0c-ec3d-4c88-9fed-bd20d0435f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:53.174235 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d00c0c-ec3d-4c88-9fed-bd20d0435f59 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:53.174562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d00c0c-ec3d-4c88-9fed-bd20d0435f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:53.174842 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d00c0c-ec3d-4c88-9fed-bd20d0435f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:53.176604 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d00c0c-ec3d-4c88-9fed-bd20d0435f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:53.182007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d00c0c-ec3d-4c88-9fed-bd20d0435f59 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:53.191162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:53.221565 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e4af5-f70e-48f6-ade0-a389aad21d08 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:53.232805 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 681/1374] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:53.236478 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e4af5-f70e-48f6-ade0-a389aad21d08 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:53.269058 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11b937-5475-4b16-a6b3-c5acb5f82107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:53.271354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11b937-5475-4b16-a6b3-c5acb5f82107 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:53.274496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11b937-5475-4b16-a6b3-c5acb5f82107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:53.280812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11b937-5475-4b16-a6b3-c5acb5f82107 admin admin] PATH_INFO: `/v3/projects/9472992c10704cd88adec4abf465bbea/users/438983af2bec4112aa2d72f4bea5560c/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:53.368960 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11b937-5475-4b16-a6b3-c5acb5f82107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9472992c10704cd88adec4abf465bbea, user_id=438983af2bec4112aa2d72f4bea5560c, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:53.386108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11b937-5475-4b16-a6b3-c5acb5f82107 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:53.447673 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 694/1375] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:53 2022] PUT /identity/v3/projects/9472992c10704cd88adec4abf465bbea/users/438983af2bec4112aa2d72f4bea5560c/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:53.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209a271-7e8f-494c-8eec-cc28c3da66b6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:53.462372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209a271-7e8f-494c-8eec-cc28c3da66b6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:53.462673 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209a271-7e8f-494c-8eec-cc28c3da66b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:53.550846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1209a271-7e8f-494c-8eec-cc28c3da66b6 None None] MFA Rules not processed for user `438983af2bec4112aa2d72f4bea5560c`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:53.553134 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1209a271-7e8f-494c-8eec-cc28c3da66b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:53.824289 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 682/1376] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:53.828768 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11b937-5475-4b16-a6b3-c5acb5f82107 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:53.888534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:53.981270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11b937-5475-4b16-a6b3-c5acb5f82107 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:53.989985 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e0c62-8428-4b35-b00e-c495372495e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:53.992763 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e0c62-8428-4b35-b00e-c495372495e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:53.993188 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e0c62-8428-4b35-b00e-c495372495e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:53.994565 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e0c62-8428-4b35-b00e-c495372495e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:53.996164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e0c62-8428-4b35-b00e-c495372495e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:54.002335 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e0c62-8428-4b35-b00e-c495372495e1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:54.082700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 695/1377] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:11:53 2022] POST /identity/v3/projects => generated 363 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:54.087092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209a271-7e8f-494c-8eec-cc28c3da66b6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:54.120080 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209a271-7e8f-494c-8eec-cc28c3da66b6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:54.298835 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780edf54-86ef-4276-84d7-e5130a8ae765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:54.299508 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780edf54-86ef-4276-84d7-e5130a8ae765 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:54.304295 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780edf54-86ef-4276-84d7-e5130a8ae765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:54.305603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780edf54-86ef-4276-84d7-e5130a8ae765 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:54.307352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780edf54-86ef-4276-84d7-e5130a8ae765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:54.310847 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780edf54-86ef-4276-84d7-e5130a8ae765 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:54.362443 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 683/1378] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:11:54 2022] POST /identity/v3/users => generated 288 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:54.366039 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e0c62-8428-4b35-b00e-c495372495e1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:54.383895 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e0c62-8428-4b35-b00e-c495372495e1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:54.404228 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414973e6-e65a-4aa3-84e8-fd58e5b024b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:54.404897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414973e6-e65a-4aa3-84e8-fd58e5b024b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:54.409918 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414973e6-e65a-4aa3-84e8-fd58e5b024b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:54.410268 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414973e6-e65a-4aa3-84e8-fd58e5b024b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:54.411714 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414973e6-e65a-4aa3-84e8-fd58e5b024b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:54.414658 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414973e6-e65a-4aa3-84e8-fd58e5b024b0 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:54.461615 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 696/1379] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:54.462416 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780edf54-86ef-4276-84d7-e5130a8ae765 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:54.478816 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780edf54-86ef-4276-84d7-e5130a8ae765 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:54.497612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1b3c4-d636-414d-8a76-d3f1f703ad3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:54.499619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1b3c4-d636-414d-8a76-d3f1f703ad3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:54.500065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1b3c4-d636-414d-8a76-d3f1f703ad3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:54.500354 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1b3c4-d636-414d-8a76-d3f1f703ad3a admin admin] PATH_INFO: `/v3/projects/61b04306c0bf4994b99986bc33a99ba4/users/7bd3e9df98f643948c0d9abf05fea455/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:54.557994 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1b3c4-d636-414d-8a76-d3f1f703ad3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=61b04306c0bf4994b99986bc33a99ba4, user_id=7bd3e9df98f643948c0d9abf05fea455, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:54.568552 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1b3c4-d636-414d-8a76-d3f1f703ad3a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:54.615949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 684/1380] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:54 2022] PUT /identity/v3/projects/61b04306c0bf4994b99986bc33a99ba4/users/7bd3e9df98f643948c0d9abf05fea455/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:54.621501 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414973e6-e65a-4aa3-84e8-fd58e5b024b0 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:54.654662 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414973e6-e65a-4aa3-84e8-fd58e5b024b0 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:54.784079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1b3c4-d636-414d-8a76-d3f1f703ad3a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:54.786132 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-81b1b3c4-d636-414d-8a76-d3f1f703ad3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:54.806994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08407729-fd33-40e0-bbea-36904aa2deb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:54.809143 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08407729-fd33-40e0-bbea-36904aa2deb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:54.810063 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08407729-fd33-40e0-bbea-36904aa2deb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:54.810630 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08407729-fd33-40e0-bbea-36904aa2deb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:54.813804 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08407729-fd33-40e0-bbea-36904aa2deb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:54.818350 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08407729-fd33-40e0-bbea-36904aa2deb0 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:54.820179 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1b3c4-d636-414d-8a76-d3f1f703ad3a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:54.840860 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c753fc-4f3c-4270-8584-a951f726aed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:54.842048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c753fc-4f3c-4270-8584-a951f726aed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:54.842783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c753fc-4f3c-4270-8584-a951f726aed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:54.844429 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c753fc-4f3c-4270-8584-a951f726aed6 admin admin] PATH_INFO: `/v3/projects/1c26c9ea1fc64309b38b2d3345140d04` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:54.847755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c753fc-4f3c-4270-8584-a951f726aed6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c26c9ea1fc64309b38b2d3345140d04)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:54.851582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 697/1381] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:54.854137 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08407729-fd33-40e0-bbea-36904aa2deb0 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:54.854324 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c753fc-4f3c-4270-8584-a951f726aed6 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:54.894711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08407729-fd33-40e0-bbea-36904aa2deb0 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:54.908973 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193358a-bb3d-457c-8165-31a8c9628d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:54.909922 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193358a-bb3d-457c-8165-31a8c9628d78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:54.912171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193358a-bb3d-457c-8165-31a8c9628d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:54.912638 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193358a-bb3d-457c-8165-31a8c9628d78 admin admin] PATH_INFO: `/v3/projects/61b04306c0bf4994b99986bc33a99ba4/users/7bd3e9df98f643948c0d9abf05fea455/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:54.917481 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193358a-bb3d-457c-8165-31a8c9628d78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61b04306c0bf4994b99986bc33a99ba4, user_id=7bd3e9df98f643948c0d9abf05fea455, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:54.923531 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193358a-bb3d-457c-8165-31a8c9628d78 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:54.945793 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:54.948345 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e4c753fc-4f3c-4270-8584-a951f726aed6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c26c9ea1fc64309b38b2d3345140d04', 'request_id': 'req-e4c753fc-4f3c-4270-8584-a951f726aed6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:54.954256 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 698/1382] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:54 2022] PUT /identity/v3/projects/61b04306c0bf4994b99986bc33a99ba4/users/7bd3e9df98f643948c0d9abf05fea455/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:54.967445 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285cdc5-2d5b-4ab9-92cc-e337aa3b9bf3 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:54.968158 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285cdc5-2d5b-4ab9-92cc-e337aa3b9bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:54.968598 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285cdc5-2d5b-4ab9-92cc-e337aa3b9bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:54.970428 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e4c753fc-4f3c-4270-8584-a951f726aed6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c26c9ea1fc64309b38b2d3345140d04', 'request_id': 'req-e4c753fc-4f3c-4270-8584-a951f726aed6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:54.978493 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e4c753fc-4f3c-4270-8584-a951f726aed6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c26c9ea1fc64309b38b2d3345140d04', 'request_id': 'req-e4c753fc-4f3c-4270-8584-a951f726aed6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:55.001691 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e4c753fc-4f3c-4270-8584-a951f726aed6 admin admin] The token cache is being invalidate because project 1c26c9ea1fc64309b38b2d3345140d04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:11:55.002565 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e4c753fc-4f3c-4270-8584-a951f726aed6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:55.003499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e4c753fc-4f3c-4270-8584-a951f726aed6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:55.005615 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 685/1383] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:11:54 2022] DELETE /identity/v3/projects/1c26c9ea1fc64309b38b2d3345140d04 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:55.010311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c753fc-4f3c-4270-8584-a951f726aed6 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:55.013046 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e4c753fc-4f3c-4270-8584-a951f726aed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:55.053612 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0285cdc5-2d5b-4ab9-92cc-e337aa3b9bf3 None None] MFA Rules not processed for user `7bd3e9df98f643948c0d9abf05fea455`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:55.055168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0285cdc5-2d5b-4ab9-92cc-e337aa3b9bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:55.133240 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c753fc-4f3c-4270-8584-a951f726aed6 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:55.200819 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 699/1384] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:54 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:11:55.214732 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285cdc5-2d5b-4ab9-92cc-e337aa3b9bf3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:55.218397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0285cdc5-2d5b-4ab9-92cc-e337aa3b9bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:55.227352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:55.231768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296d4a9-fd7c-4954-a0fd-11fdedc1f486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:55.232628 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296d4a9-fd7c-4954-a0fd-11fdedc1f486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:55.233250 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296d4a9-fd7c-4954-a0fd-11fdedc1f486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:55.233675 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296d4a9-fd7c-4954-a0fd-11fdedc1f486 admin admin] PATH_INFO: `/v3/users/3f34e58670c449c19718caba423ac5d7` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:55.235386 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296d4a9-fd7c-4954-a0fd-11fdedc1f486 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f34e58670c449c19718caba423ac5d7)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:55.237635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296d4a9-fd7c-4954-a0fd-11fdedc1f486 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:55.368453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b296d4a9-fd7c-4954-a0fd-11fdedc1f486 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f34e58670c449c19718caba423ac5d7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:55.388829 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b296d4a9-fd7c-4954-a0fd-11fdedc1f486 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f34e58670c449c19718caba423ac5d7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:55.389841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b296d4a9-fd7c-4954-a0fd-11fdedc1f486 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f34e58670c449c19718caba423ac5d7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:55.391093 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285cdc5-2d5b-4ab9-92cc-e337aa3b9bf3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:55.401977 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407b348-5782-4964-af41-2716d84c76c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:55.402890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407b348-5782-4964-af41-2716d84c76c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:55.405678 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407b348-5782-4964-af41-2716d84c76c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:55.408072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407b348-5782-4964-af41-2716d84c76c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:55.419687 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b296d4a9-fd7c-4954-a0fd-11fdedc1f486 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f34e58670c449c19718caba423ac5d7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:55.421197 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9407b348-5782-4964-af41-2716d84c76c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:55.439872 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407b348-5782-4964-af41-2716d84c76c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:55.441702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407b348-5782-4964-af41-2716d84c76c4 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:55.447122 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b296d4a9-fd7c-4954-a0fd-11fdedc1f486 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f34e58670c449c19718caba423ac5d7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:55.457158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 686/1385] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:11:55 2022] DELETE /identity/v3/users/3f34e58670c449c19718caba423ac5d7 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:55.465753 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 700/1386] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:55.613402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296d4a9-fd7c-4954-a0fd-11fdedc1f486 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:55.615689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b296d4a9-fd7c-4954-a0fd-11fdedc1f486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:55.623868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:55.752505 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296d4a9-fd7c-4954-a0fd-11fdedc1f486 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:55.759257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e405a-e8c9-4d3d-8ab7-1cf02fcf34b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:55.759988 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e405a-e8c9-4d3d-8ab7-1cf02fcf34b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:55.760314 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e405a-e8c9-4d3d-8ab7-1cf02fcf34b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:55.760767 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e405a-e8c9-4d3d-8ab7-1cf02fcf34b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:55.764027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-103e405a-e8c9-4d3d-8ab7-1cf02fcf34b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:55.865092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e405a-e8c9-4d3d-8ab7-1cf02fcf34b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:55.867441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e405a-e8c9-4d3d-8ab7-1cf02fcf34b1 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:55.906604 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 687/1387] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:56.045768 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb9173-eba3-4c49-87e1-c399e051fb70 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:56.046418 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb9173-eba3-4c49-87e1-c399e051fb70 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:56.046921 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb9173-eba3-4c49-87e1-c399e051fb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:56.136614 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7dcb9173-eba3-4c49-87e1-c399e051fb70 None None] MFA Rules not processed for user `9c1787a9ef7c44e28076aaa80c0152fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:56.138024 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7dcb9173-eba3-4c49-87e1-c399e051fb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:56.272330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee19cf-9ad2-4e43-94a0-16824fe219fe None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:56.273494 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee19cf-9ad2-4e43-94a0-16824fe219fe None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:56.274446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee19cf-9ad2-4e43-94a0-16824fe219fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:56.360459 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b7ee19cf-9ad2-4e43-94a0-16824fe219fe None None] MFA Rules not processed for user `ce25db99a7b14d0bbaca41fff80e3da9`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:56.364167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b7ee19cf-9ad2-4e43-94a0-16824fe219fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:56.374721 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 701/1388] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:56.385370 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25228-11de-4209-8bfc-5a23e796bf7e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:56.386011 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25228-11de-4209-8bfc-5a23e796bf7e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:56.386407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25228-11de-4209-8bfc-5a23e796bf7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:56.435957 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-21d25228-11de-4209-8bfc-5a23e796bf7e None None] MFA Rules not processed for user `9c1787a9ef7c44e28076aaa80c0152fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:56.439014 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-21d25228-11de-4209-8bfc-5a23e796bf7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:56.481900 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 702/1389] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:56.483730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052a71a-68c6-4ad3-bc6e-34ccf735ecfa None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:56.484264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052a71a-68c6-4ad3-bc6e-34ccf735ecfa None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:56.488055 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052a71a-68c6-4ad3-bc6e-34ccf735ecfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:56.556190 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e052a71a-68c6-4ad3-bc6e-34ccf735ecfa None None] MFA Rules not processed for user `37e6d36d529146299970874660e92e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:56.558735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e052a71a-68c6-4ad3-bc6e-34ccf735ecfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:56.595331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 688/1390] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:56.596328 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee19cf-9ad2-4e43-94a0-16824fe219fe None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:56.599611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b7ee19cf-9ad2-4e43-94a0-16824fe219fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:56.696188 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 703/1391] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:56.697185 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052a71a-68c6-4ad3-bc6e-34ccf735ecfa None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:56.700269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e052a71a-68c6-4ad3-bc6e-34ccf735ecfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:56.714449 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:56.809657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee19cf-9ad2-4e43-94a0-16824fe219fe None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:56.824763 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25c6cc-50aa-42d6-be54-2076b0ddaa93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:56.825413 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25c6cc-50aa-42d6-be54-2076b0ddaa93 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:56.825697 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25c6cc-50aa-42d6-be54-2076b0ddaa93 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:56.825970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25c6cc-50aa-42d6-be54-2076b0ddaa93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:56.828463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:56.848725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25c6cc-50aa-42d6-be54-2076b0ddaa93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:56.850278 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25c6cc-50aa-42d6-be54-2076b0ddaa93 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:56.875070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 689/1392] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:56.912651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052a71a-68c6-4ad3-bc6e-34ccf735ecfa None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:56.927194 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aaf1ed-66e1-4ed5-8522-2ec987800fbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:56.927854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaf1ed-66e1-4ed5-8522-2ec987800fbc service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:56.928126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaf1ed-66e1-4ed5-8522-2ec987800fbc service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:56.928405 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaf1ed-66e1-4ed5-8522-2ec987800fbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:56.956565 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aaf1ed-66e1-4ed5-8522-2ec987800fbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:56.958817 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aaf1ed-66e1-4ed5-8522-2ec987800fbc service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:56.995826 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 704/1393] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:57.482999 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25c6cc-50aa-42d6-be54-2076b0ddaa93 service nova] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:57.484631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3a25c6cc-50aa-42d6-be54-2076b0ddaa93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:57.629461 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25c6cc-50aa-42d6-be54-2076b0ddaa93 service nova] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:57.639987 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263fac3-4e5e-4182-96dd-a93c9593a3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:57.640925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263fac3-4e5e-4182-96dd-a93c9593a3b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:57.641366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263fac3-4e5e-4182-96dd-a93c9593a3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:57.641785 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263fac3-4e5e-4182-96dd-a93c9593a3b1 admin admin] PATH_INFO: `/v3/projects/8e4b58b189e2470dabf95cbdce14bd08` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:57.643868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263fac3-4e5e-4182-96dd-a93c9593a3b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e4b58b189e2470dabf95cbdce14bd08)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:57.646356 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263fac3-4e5e-4182-96dd-a93c9593a3b1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:57.773670 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2263fac3-4e5e-4182-96dd-a93c9593a3b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e4b58b189e2470dabf95cbdce14bd08', 'request_id': 'req-2263fac3-4e5e-4182-96dd-a93c9593a3b1'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:57.783861 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2263fac3-4e5e-4182-96dd-a93c9593a3b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e4b58b189e2470dabf95cbdce14bd08', 'request_id': 'req-2263fac3-4e5e-4182-96dd-a93c9593a3b1'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:57.799460 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2263fac3-4e5e-4182-96dd-a93c9593a3b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e4b58b189e2470dabf95cbdce14bd08', 'request_id': 'req-2263fac3-4e5e-4182-96dd-a93c9593a3b1'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:57.821864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2263fac3-4e5e-4182-96dd-a93c9593a3b1 admin admin] The token cache is being invalidate because project 8e4b58b189e2470dabf95cbdce14bd08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:11:57.823303 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2263fac3-4e5e-4182-96dd-a93c9593a3b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:57.824943 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2263fac3-4e5e-4182-96dd-a93c9593a3b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:11:57.834315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 690/1394] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:11:57 2022] DELETE /identity/v3/projects/8e4b58b189e2470dabf95cbdce14bd08 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:57.836300 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b39ae7-1446-4ce3-b55d-9a38cae47262 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:57.836767 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b39ae7-1446-4ce3-b55d-9a38cae47262 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:57.836843 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b39ae7-1446-4ce3-b55d-9a38cae47262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:57.868050 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-b5b39ae7-1446-4ce3-b55d-9a38cae47262 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:57.869435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b5b39ae7-1446-4ce3-b55d-9a38cae47262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:57.975871 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 705/1395] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:57.977872 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852daf4c-8afe-4821-9c5c-d64cb0e476bb None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:57.978357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852daf4c-8afe-4821-9c5c-d64cb0e476bb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:57.978749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852daf4c-8afe-4821-9c5c-d64cb0e476bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:58.005751 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-852daf4c-8afe-4821-9c5c-d64cb0e476bb None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:58.008087 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-852daf4c-8afe-4821-9c5c-d64cb0e476bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:58.033534 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 691/1396] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:58.036063 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b39ae7-1446-4ce3-b55d-9a38cae47262 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:58.050390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b39ae7-1446-4ce3-b55d-9a38cae47262 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:58.062949 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cfa6e-e383-4b24-b5bf-884b64d3acec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:58.065340 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cfa6e-e383-4b24-b5bf-884b64d3acec admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:58.068135 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cfa6e-e383-4b24-b5bf-884b64d3acec admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:58.068547 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cfa6e-e383-4b24-b5bf-884b64d3acec admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:58.069562 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4cfa6e-e383-4b24-b5bf-884b64d3acec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:11:58.070012 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4cfa6e-e383-4b24-b5bf-884b64d3acec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:58.072456 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4cfa6e-e383-4b24-b5bf-884b64d3acec admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:58.114701 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 706/1397] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:11:58.116861 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852daf4c-8afe-4821-9c5c-d64cb0e476bb None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:58.129237 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852daf4c-8afe-4821-9c5c-d64cb0e476bb None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:58.135637 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a32fed-3f49-4fab-8373-bc7f1072881b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:58.136177 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a32fed-3f49-4fab-8373-bc7f1072881b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:58.138589 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a32fed-3f49-4fab-8373-bc7f1072881b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:58.144539 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5356872-764a-4805-aa89-829a0a178620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:58.145176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5356872-764a-4805-aa89-829a0a178620 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:58.145454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5356872-764a-4805-aa89-829a0a178620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:58.149954 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5356872-764a-4805-aa89-829a0a178620 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:58.151010 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5356872-764a-4805-aa89-829a0a178620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:58.155032 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5356872-764a-4805-aa89-829a0a178620 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:58.206182 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-16a32fed-3f49-4fab-8373-bc7f1072881b None None] MFA Rules not processed for user `1c786544e8fb4e8e8c3a4990a0f36725`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:58.210854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-16a32fed-3f49-4fab-8373-bc7f1072881b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:58.224433 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 692/1398] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:11:58 2022] POST /identity/v3/projects => generated 363 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:58.230500 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5356872-764a-4805-aa89-829a0a178620 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:58.361037 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 707/1399] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:58.369935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c37bf-962c-4466-b744-4270086980b7 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:58.370222 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c37bf-962c-4466-b744-4270086980b7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:58.370497 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c37bf-962c-4466-b744-4270086980b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:58.389982 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:58.468398 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6b7c37bf-962c-4466-b744-4270086980b7 None None] MFA Rules not processed for user `4443729a5a5341b6ad64f6fa131a5a54`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:58.469444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6b7c37bf-962c-4466-b744-4270086980b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:58.479771 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5356872-764a-4805-aa89-829a0a178620 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:58.490613 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae612d-3555-4edb-9f37-823560313904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:58.495267 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae612d-3555-4edb-9f37-823560313904 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:58.495551 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae612d-3555-4edb-9f37-823560313904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:58.495825 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae612d-3555-4edb-9f37-823560313904 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:58.496477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae612d-3555-4edb-9f37-823560313904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:58.501723 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae612d-3555-4edb-9f37-823560313904 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:58.534299 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:11:58.551256 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 693/1400] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:11:58 2022] POST /identity/v3/users => generated 282 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:58.552810 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ff504-c5bf-4a50-862f-e9a5b349ffc2 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:58.553107 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ff504-c5bf-4a50-862f-e9a5b349ffc2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:58.553383 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ff504-c5bf-4a50-862f-e9a5b349ffc2 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:58.564484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 694/1401] 10.222.0.27 () {54 vars in 912 bytes} [Mon Oct 17 03:11:58 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:11:58.566481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6238c-b604-40bb-a124-e095f315df62 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:58.567165 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6238c-b604-40bb-a124-e095f315df62 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:58.567389 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6238c-b604-40bb-a124-e095f315df62 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:58.570570 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 695/1402] 10.222.0.27 () {54 vars in 912 bytes} [Mon Oct 17 03:11:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:11:58.571457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6238c-b604-40bb-a124-e095f315df62 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:58.590442 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6238c-b604-40bb-a124-e095f315df62 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:58.612507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537aa33e-c506-4050-a0b2-467b0956ce61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:58.613207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537aa33e-c506-4050-a0b2-467b0956ce61 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:58.613499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537aa33e-c506-4050-a0b2-467b0956ce61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:58.613779 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537aa33e-c506-4050-a0b2-467b0956ce61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:58.614443 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537aa33e-c506-4050-a0b2-467b0956ce61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:58.618940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537aa33e-c506-4050-a0b2-467b0956ce61 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:58.657095 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 696/1403] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:58.660104 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c13f19-908d-469f-a814-6f924c3d1018 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:58.660104 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c13f19-908d-469f-a814-6f924c3d1018 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:58.660104 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c13f19-908d-469f-a814-6f924c3d1018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:58.666805 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 708/1404] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:11:58.668447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2d334-4459-4d18-b1ed-e36ef204fb35 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:58.668821 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2d334-4459-4d18-b1ed-e36ef204fb35 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:58.669306 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2d334-4459-4d18-b1ed-e36ef204fb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:58.739234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-90f2d334-4459-4d18-b1ed-e36ef204fb35 None None] MFA Rules not processed for user `526933e6ad004493b071f5da5caa09a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:58.740517 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-90f2d334-4459-4d18-b1ed-e36ef204fb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:58.770140 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-84c13f19-908d-469f-a814-6f924c3d1018 None None] MFA Rules not processed for user `526933e6ad004493b071f5da5caa09a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:58.771416 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-84c13f19-908d-469f-a814-6f924c3d1018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:58.846399 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 709/1405] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Oct 17 03:11:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:58.848055 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2d334-4459-4d18-b1ed-e36ef204fb35 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:58.871613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2d334-4459-4d18-b1ed-e36ef204fb35 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:58.886779 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a7663-8cdc-40ed-8256-1b02075fbaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:58.906634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a7663-8cdc-40ed-8256-1b02075fbaf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:58.912401 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a7663-8cdc-40ed-8256-1b02075fbaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:58.912595 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a7663-8cdc-40ed-8256-1b02075fbaf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:58.912993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 697/1406] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Oct 17 03:11:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:58.914352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483589d7-8469-4ad5-b23b-de6b50a462f1 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:58.914662 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483589d7-8469-4ad5-b23b-de6b50a462f1 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:58.915220 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483589d7-8469-4ad5-b23b-de6b50a462f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:58.916813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a7663-8cdc-40ed-8256-1b02075fbaf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:58.926805 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a7663-8cdc-40ed-8256-1b02075fbaf0 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:58.959088 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 710/1407] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:11:58.960277 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a7663-8cdc-40ed-8256-1b02075fbaf0 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:58.967087 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d84a7663-8cdc-40ed-8256-1b02075fbaf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:58.989828 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a7663-8cdc-40ed-8256-1b02075fbaf0 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:58.999256 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69916b-7109-410b-9d61-6c76ddf698c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:58.999903 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69916b-7109-410b-9d61-6c76ddf698c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:59.000181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69916b-7109-410b-9d61-6c76ddf698c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:59.000462 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69916b-7109-410b-9d61-6c76ddf698c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:59.014457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69916b-7109-410b-9d61-6c76ddf698c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:59.016079 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69916b-7109-410b-9d61-6c76ddf698c4 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:59.026562 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 711/1408] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:59.027931 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69916b-7109-410b-9d61-6c76ddf698c4 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:59.030584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5f69916b-7109-410b-9d61-6c76ddf698c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:59.039552 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-483589d7-8469-4ad5-b23b-de6b50a462f1 None None] MFA Rules not processed for user `4443729a5a5341b6ad64f6fa131a5a54`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:59.040797 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-483589d7-8469-4ad5-b23b-de6b50a462f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:59.062002 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69916b-7109-410b-9d61-6c76ddf698c4 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:59.069801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4279199-b0eb-4068-809b-cba6883b52d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:59.070893 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4279199-b0eb-4068-809b-cba6883b52d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:59.072145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4279199-b0eb-4068-809b-cba6883b52d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:59.072440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4279199-b0eb-4068-809b-cba6883b52d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:59.091359 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4279199-b0eb-4068-809b-cba6883b52d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:59.093584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4279199-b0eb-4068-809b-cba6883b52d5 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:59.101668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 698/1409] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:59.106757 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483589d7-8469-4ad5-b23b-de6b50a462f1 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:59.111685 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 712/1410] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:11:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:11:59.134648 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483589d7-8469-4ad5-b23b-de6b50a462f1 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:59.152993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fb7c6-9d0d-434d-ac75-a551ca42badc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:59.153636 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fb7c6-9d0d-434d-ac75-a551ca42badc admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:59.153924 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fb7c6-9d0d-434d-ac75-a551ca42badc admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:59.154203 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fb7c6-9d0d-434d-ac75-a551ca42badc admin admin] PATH_INFO: `/v3/projects/add0636e6b964fffbf0bb1c395d6f95b/users/987a816bd7f54da0bd0cfffb5b5eaf78/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:59.216026 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fb7c6-9d0d-434d-ac75-a551ca42badc admin admin] RBAC: Authorizing `identity:create_grant(project_id=add0636e6b964fffbf0bb1c395d6f95b, user_id=987a816bd7f54da0bd0cfffb5b5eaf78, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:59.218707 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fb7c6-9d0d-434d-ac75-a551ca42badc admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:59.281335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 699/1411] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:11:59 2022] PUT /identity/v3/projects/add0636e6b964fffbf0bb1c395d6f95b/users/987a816bd7f54da0bd0cfffb5b5eaf78/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:11:59.294277 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b22f9-0cd0-45af-9d9d-2119f1621fa9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:59.294277 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b22f9-0cd0-45af-9d9d-2119f1621fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:59.294629 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b22f9-0cd0-45af-9d9d-2119f1621fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:59.378376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-766b22f9-0cd0-45af-9d9d-2119f1621fa9 None None] MFA Rules not processed for user `987a816bd7f54da0bd0cfffb5b5eaf78`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:11:59.380856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-766b22f9-0cd0-45af-9d9d-2119f1621fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:11:59.553222 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 713/1412] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:11:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:11:59.567388 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fb7c6-9d0d-434d-ac75-a551ca42badc admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:59.793342 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fb7c6-9d0d-434d-ac75-a551ca42badc admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:59.803221 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edc5be-dcef-4a56-9b74-533fad359d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:11:59.804126 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edc5be-dcef-4a56-9b74-533fad359d8f admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:11:59.804800 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edc5be-dcef-4a56-9b74-533fad359d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:11:59.805289 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edc5be-dcef-4a56-9b74-533fad359d8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:11:59.806087 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edc5be-dcef-4a56-9b74-533fad359d8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:11:59.808211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edc5be-dcef-4a56-9b74-533fad359d8f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:11:59.895275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 700/1413] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:11:59 2022] POST /identity/v3/projects => generated 363 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:11:59.899318 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b22f9-0cd0-45af-9d9d-2119f1621fa9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:11:59.932730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b22f9-0cd0-45af-9d9d-2119f1621fa9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:11:59.973968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:00.142673 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a80bb0-721a-45f1-974b-66d33b1d78b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:00.151771 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a80bb0-721a-45f1-974b-66d33b1d78b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:00.152305 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a80bb0-721a-45f1-974b-66d33b1d78b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:00.152590 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a80bb0-721a-45f1-974b-66d33b1d78b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:00.153286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a80bb0-721a-45f1-974b-66d33b1d78b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:00.163799 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a80bb0-721a-45f1-974b-66d33b1d78b9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:00.218235 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 714/1414] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:11:59 2022] POST /identity/v3/users => generated 288 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:00.219475 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edc5be-dcef-4a56-9b74-533fad359d8f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:00.238150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edc5be-dcef-4a56-9b74-533fad359d8f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:00.239955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a80bb0-721a-45f1-974b-66d33b1d78b9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:00.245974 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-65a80bb0-721a-45f1-974b-66d33b1d78b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:00.266330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead29d6e-5f9d-4f22-b8c4-99a972c880b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:00.267008 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead29d6e-5f9d-4f22-b8c4-99a972c880b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:00.267304 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead29d6e-5f9d-4f22-b8c4-99a972c880b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:00.267545 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead29d6e-5f9d-4f22-b8c4-99a972c880b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:00.268208 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead29d6e-5f9d-4f22-b8c4-99a972c880b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:00.269893 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead29d6e-5f9d-4f22-b8c4-99a972c880b1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:00.289270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:00.329281 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 701/1415] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:00.340120 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead29d6e-5f9d-4f22-b8c4-99a972c880b1 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:00.380720 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead29d6e-5f9d-4f22-b8c4-99a972c880b1 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:00.410462 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef995df7-37f8-47cb-bb30-836dc24d3075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:00.411210 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef995df7-37f8-47cb-bb30-836dc24d3075 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:00.411497 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef995df7-37f8-47cb-bb30-836dc24d3075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:00.411772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef995df7-37f8-47cb-bb30-836dc24d3075 admin admin] PATH_INFO: `/v3/projects/e14e704856244b2fa08d1d5513f301fc/users/685e4589907346109bf623798d50a6e1/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:00.470457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef995df7-37f8-47cb-bb30-836dc24d3075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e14e704856244b2fa08d1d5513f301fc, user_id=685e4589907346109bf623798d50a6e1, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:00.473251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef995df7-37f8-47cb-bb30-836dc24d3075 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:00.528968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a80bb0-721a-45f1-974b-66d33b1d78b9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:00.536799 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 702/1416] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:00 2022] PUT /identity/v3/projects/e14e704856244b2fa08d1d5513f301fc/users/685e4589907346109bf623798d50a6e1/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:00.540883 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef995df7-37f8-47cb-bb30-836dc24d3075 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:00.544279 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ef995df7-37f8-47cb-bb30-836dc24d3075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:00.842473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9faefb-c0c2-44fb-8e64-a1d4a5bfc06f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:00.845348 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9faefb-c0c2-44fb-8e64-a1d4a5bfc06f service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:00.845642 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9faefb-c0c2-44fb-8e64-a1d4a5bfc06f service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:00.846580 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9faefb-c0c2-44fb-8e64-a1d4a5bfc06f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:00.898312 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef995df7-37f8-47cb-bb30-836dc24d3075 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:00.916015 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e380f-3c8e-4d6e-8fa1-a675d9dfdf52 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cc2655cc32e4d63a7000ef41bcbcfea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:00.917268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e380f-3c8e-4d6e-8fa1-a675d9dfdf52 service placement] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:00.918512 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e380f-3c8e-4d6e-8fa1-a675d9dfdf52 service placement] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:00.919971 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e380f-3c8e-4d6e-8fa1-a675d9dfdf52 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:00.924614 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bc4e380f-3c8e-4d6e-8fa1-a675d9dfdf52 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:00.958298 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e380f-3c8e-4d6e-8fa1-a675d9dfdf52 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:00.963740 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e380f-3c8e-4d6e-8fa1-a675d9dfdf52 service placement] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:00.989816 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:01.010902 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 703/1417] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:01.012318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e380f-3c8e-4d6e-8fa1-a675d9dfdf52 service placement] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:01.045590 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9faefb-c0c2-44fb-8e64-a1d4a5bfc06f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:01.047706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9faefb-c0c2-44fb-8e64-a1d4a5bfc06f service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:01.085863 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 715/1418] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 846 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:01.091820 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9faefb-c0c2-44fb-8e64-a1d4a5bfc06f service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:01.095117 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7c9faefb-c0c2-44fb-8e64-a1d4a5bfc06f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:01.129571 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e380f-3c8e-4d6e-8fa1-a675d9dfdf52 service placement] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:01.136816 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c05c7-a041-47c9-8645-473c9fdee56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:01.137655 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c05c7-a041-47c9-8645-473c9fdee56a admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:01.138824 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9faefb-c0c2-44fb-8e64-a1d4a5bfc06f service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:01.139113 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c05c7-a041-47c9-8645-473c9fdee56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:01.139661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c05c7-a041-47c9-8645-473c9fdee56a admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:01.140570 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c05c7-a041-47c9-8645-473c9fdee56a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:01.142833 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c05c7-a041-47c9-8645-473c9fdee56a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:01.154818 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae12ebf-7af5-43a1-aa72-c63311424730 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:01.156814 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae12ebf-7af5-43a1-aa72-c63311424730 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:01.158052 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae12ebf-7af5-43a1-aa72-c63311424730 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:01.158580 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae12ebf-7af5-43a1-aa72-c63311424730 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:01.162434 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 704/1419] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:01.163138 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb40038-cf2a-4989-b068-18f3d3197526 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:01.163454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb40038-cf2a-4989-b068-18f3d3197526 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:01.164892 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb40038-cf2a-4989-b068-18f3d3197526 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:01.171235 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 705/1420] 10.222.0.27 () {54 vars in 961 bytes} [Mon Oct 17 03:12:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:12:01.172649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb40038-cf2a-4989-b068-18f3d3197526 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:01.177683 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-adb40038-cf2a-4989-b068-18f3d3197526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:01.222275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb40038-cf2a-4989-b068-18f3d3197526 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:01.235605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c23b3e-6337-4900-9de0-a26181034482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:01.236353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c23b3e-6337-4900-9de0-a26181034482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:01.242444 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c23b3e-6337-4900-9de0-a26181034482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:01.242827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c23b3e-6337-4900-9de0-a26181034482 admin admin] PATH_INFO: `/v3/users/438983af2bec4112aa2d72f4bea5560c` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:01.247562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c23b3e-6337-4900-9de0-a26181034482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=438983af2bec4112aa2d72f4bea5560c)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:01.251227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c23b3e-6337-4900-9de0-a26181034482 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:01.367191 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-96c23b3e-6337-4900-9de0-a26181034482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '438983af2bec4112aa2d72f4bea5560c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:01.376558 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:01.386335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-96c23b3e-6337-4900-9de0-a26181034482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '438983af2bec4112aa2d72f4bea5560c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:01.390385 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-96c23b3e-6337-4900-9de0-a26181034482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '438983af2bec4112aa2d72f4bea5560c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:01.409168 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-96c23b3e-6337-4900-9de0-a26181034482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '438983af2bec4112aa2d72f4bea5560c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:01.435608 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-96c23b3e-6337-4900-9de0-a26181034482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '438983af2bec4112aa2d72f4bea5560c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:01.452328 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae12ebf-7af5-43a1-aa72-c63311424730 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:01.454818 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae12ebf-7af5-43a1-aa72-c63311424730 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:01.455909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 706/1421] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:01 2022] DELETE /identity/v3/users/438983af2bec4112aa2d72f4bea5560c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:01.457045 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c23b3e-6337-4900-9de0-a26181034482 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:01.466301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-96c23b3e-6337-4900-9de0-a26181034482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:01.505537 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 716/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:01.519481 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243d9ad-7d72-4ff5-a129-803ac4d5f3f4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:01.523205 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243d9ad-7d72-4ff5-a129-803ac4d5f3f4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:01.524709 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243d9ad-7d72-4ff5-a129-803ac4d5f3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:01.637417 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d243d9ad-7d72-4ff5-a129-803ac4d5f3f4 None None] MFA Rules not processed for user `4cc2655cc32e4d63a7000ef41bcbcfea`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:01.647039 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d243d9ad-7d72-4ff5-a129-803ac4d5f3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:01.711872 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c23b3e-6337-4900-9de0-a26181034482 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:01.729422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43342643-170a-4f45-a4bc-d40671501f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:01.729908 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43342643-170a-4f45-a4bc-d40671501f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:01.731500 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43342643-170a-4f45-a4bc-d40671501f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:01.732597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43342643-170a-4f45-a4bc-d40671501f02 admin admin] PATH_INFO: `/v3/projects/e14e704856244b2fa08d1d5513f301fc/users/685e4589907346109bf623798d50a6e1/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:01.739603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43342643-170a-4f45-a4bc-d40671501f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e14e704856244b2fa08d1d5513f301fc, user_id=685e4589907346109bf623798d50a6e1, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:01.760332 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43342643-170a-4f45-a4bc-d40671501f02 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:01.806194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 707/1423] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:01 2022] PUT /identity/v3/projects/e14e704856244b2fa08d1d5513f301fc/users/685e4589907346109bf623798d50a6e1/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:01.810116 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b3142-a01a-478a-8846-bce43a9cd072 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:01.810596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b3142-a01a-478a-8846-bce43a9cd072 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:01.811635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b3142-a01a-478a-8846-bce43a9cd072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:01.861826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e12b3142-a01a-478a-8846-bce43a9cd072 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:01.863932 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e12b3142-a01a-478a-8846-bce43a9cd072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:01.974849 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 717/1424] 10.222.0.27 () {58 vars in 1058 bytes} [Mon Oct 17 03:12:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:01.979245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443be02-2878-4e3f-86a0-bfb6d74dc811 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:01.979811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443be02-2878-4e3f-86a0-bfb6d74dc811 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:01.980221 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443be02-2878-4e3f-86a0-bfb6d74dc811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:02.034549 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 708/1425] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:02.036342 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f05e2-1c1b-4d6b-8aec-905c5fe81f26 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:02.036810 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f05e2-1c1b-4d6b-8aec-905c5fe81f26 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:02.039397 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f05e2-1c1b-4d6b-8aec-905c5fe81f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:02.054595 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2443be02-2878-4e3f-86a0-bfb6d74dc811 None None] MFA Rules not processed for user `9c1787a9ef7c44e28076aaa80c0152fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:02.061713 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2443be02-2878-4e3f-86a0-bfb6d74dc811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:02.136210 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f61f05e2-1c1b-4d6b-8aec-905c5fe81f26 None None] MFA Rules not processed for user `685e4589907346109bf623798d50a6e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:02.140595 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f61f05e2-1c1b-4d6b-8aec-905c5fe81f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:02.263777 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 718/1426] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:02.270576 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443be02-2878-4e3f-86a0-bfb6d74dc811 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:02.331592 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:02.368713 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:02.389442 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 709/1427] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:02.408686 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f05e2-1c1b-4d6b-8aec-905c5fe81f26 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:02.411242 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f61f05e2-1c1b-4d6b-8aec-905c5fe81f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:02.536236 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443be02-2878-4e3f-86a0-bfb6d74dc811 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:02.543531 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fcd6d-d13d-44a9-b4a2-82370984656e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cc2655cc32e4d63a7000ef41bcbcfea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:02.544817 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fcd6d-d13d-44a9-b4a2-82370984656e service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:02.545702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fcd6d-d13d-44a9-b4a2-82370984656e service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:02.546028 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fcd6d-d13d-44a9-b4a2-82370984656e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:02.549421 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3a3fcd6d-d13d-44a9-b4a2-82370984656e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:02.565113 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f05e2-1c1b-4d6b-8aec-905c5fe81f26 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:02.567606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3fcd6d-d13d-44a9-b4a2-82370984656e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:02.570198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3fcd6d-d13d-44a9-b4a2-82370984656e service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:02.591057 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58150d-0aed-467e-aa3f-f820e42e0fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:02.592904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 719/1428] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:02.593023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58150d-0aed-467e-aa3f-f820e42e0fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:02.593332 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58150d-0aed-467e-aa3f-f820e42e0fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:02.593646 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58150d-0aed-467e-aa3f-f820e42e0fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:02.594263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fcd6d-d13d-44a9-b4a2-82370984656e service placement] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:02.599325 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3a3fcd6d-d13d-44a9-b4a2-82370984656e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:02.617262 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58150d-0aed-467e-aa3f-f820e42e0fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:02.629014 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58150d-0aed-467e-aa3f-f820e42e0fd2 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:02.655536 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 710/1429] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:02.760457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fcd6d-d13d-44a9-b4a2-82370984656e service placement] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:02.771429 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8280b0-2d0a-4116-93e9-553970bd112c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:02.772160 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8280b0-2d0a-4116-93e9-553970bd112c service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:02.772457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8280b0-2d0a-4116-93e9-553970bd112c service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:02.773152 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8280b0-2d0a-4116-93e9-553970bd112c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:02.798933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8280b0-2d0a-4116-93e9-553970bd112c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:02.802736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8280b0-2d0a-4116-93e9-553970bd112c service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:02.861437 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 720/1430] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:03.899809 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58150d-0aed-467e-aa3f-f820e42e0fd2 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:03.903428 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bd58150d-0aed-467e-aa3f-f820e42e0fd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:03.950641 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58150d-0aed-467e-aa3f-f820e42e0fd2 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:03.962713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f1e6b-fb78-4bfa-9458-66c863e17531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:03.964740 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f1e6b-fb78-4bfa-9458-66c863e17531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:03.967630 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f1e6b-fb78-4bfa-9458-66c863e17531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:03.968167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f1e6b-fb78-4bfa-9458-66c863e17531 admin admin] PATH_INFO: `/v3/projects/9472992c10704cd88adec4abf465bbea` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:03.975151 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f1e6b-fb78-4bfa-9458-66c863e17531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9472992c10704cd88adec4abf465bbea)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:03.977448 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f1e6b-fb78-4bfa-9458-66c863e17531 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:04.009534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8280b0-2d0a-4116-93e9-553970bd112c service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:04.028013 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8280b0-2d0a-4116-93e9-553970bd112c service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:04.058690 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc183a-59ec-4314-bf77-c5f84c85685c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:04.059521 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc183a-59ec-4314-bf77-c5f84c85685c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:04.059895 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc183a-59ec-4314-bf77-c5f84c85685c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:04.060271 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc183a-59ec-4314-bf77-c5f84c85685c admin admin] PATH_INFO: `/v3/users/987a816bd7f54da0bd0cfffb5b5eaf78` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:04.062404 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc183a-59ec-4314-bf77-c5f84c85685c admin admin] RBAC: Authorizing `identity:delete_user(user_id=987a816bd7f54da0bd0cfffb5b5eaf78)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:04.078181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc183a-59ec-4314-bf77-c5f84c85685c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:04.160315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bc7f1e6b-fb78-4bfa-9458-66c863e17531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9472992c10704cd88adec4abf465bbea', 'request_id': 'req-bc7f1e6b-fb78-4bfa-9458-66c863e17531'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:04.178581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bc7f1e6b-fb78-4bfa-9458-66c863e17531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9472992c10704cd88adec4abf465bbea', 'request_id': 'req-bc7f1e6b-fb78-4bfa-9458-66c863e17531'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:04.227938 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bc7f1e6b-fb78-4bfa-9458-66c863e17531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9472992c10704cd88adec4abf465bbea', 'request_id': 'req-bc7f1e6b-fb78-4bfa-9458-66c863e17531'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:04.233413 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-14bc183a-59ec-4314-bf77-c5f84c85685c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '987a816bd7f54da0bd0cfffb5b5eaf78'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:04.265651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-14bc183a-59ec-4314-bf77-c5f84c85685c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '987a816bd7f54da0bd0cfffb5b5eaf78'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:04.269124 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-14bc183a-59ec-4314-bf77-c5f84c85685c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '987a816bd7f54da0bd0cfffb5b5eaf78'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:04.306597 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-14bc183a-59ec-4314-bf77-c5f84c85685c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '987a816bd7f54da0bd0cfffb5b5eaf78'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:04.319064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bc7f1e6b-fb78-4bfa-9458-66c863e17531 admin admin] The token cache is being invalidate because project 9472992c10704cd88adec4abf465bbea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:04.320446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bc7f1e6b-fb78-4bfa-9458-66c863e17531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:04.326493 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bc7f1e6b-fb78-4bfa-9458-66c863e17531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:04.328819 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 711/1431] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:03 2022] DELETE /identity/v3/projects/9472992c10704cd88adec4abf465bbea => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:04.337818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002db43-677d-4078-a7b2-6a49b830557d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:04.338496 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002db43-677d-4078-a7b2-6a49b830557d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:04.340281 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002db43-677d-4078-a7b2-6a49b830557d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:04.386079 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-14bc183a-59ec-4314-bf77-c5f84c85685c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '987a816bd7f54da0bd0cfffb5b5eaf78'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:04.414637 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 721/1432] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:04 2022] DELETE /identity/v3/users/987a816bd7f54da0bd0cfffb5b5eaf78 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:04.417656 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc183a-59ec-4314-bf77-c5f84c85685c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:04.431936 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-14bc183a-59ec-4314-bf77-c5f84c85685c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:04.434054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9002db43-677d-4078-a7b2-6a49b830557d None None] MFA Rules not processed for user `ce25db99a7b14d0bbaca41fff80e3da9`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:04.441484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9002db43-677d-4078-a7b2-6a49b830557d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:04.557702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:04.615599 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 712/1433] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:04.626420 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816520b-b7a0-4f44-8460-4305800ba35e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:04.626707 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816520b-b7a0-4f44-8460-4305800ba35e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:04.630757 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816520b-b7a0-4f44-8460-4305800ba35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:04.676494 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc183a-59ec-4314-bf77-c5f84c85685c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:04.688243 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624bb5a-876d-4b7a-8a7a-54d44d840758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:04.690891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624bb5a-876d-4b7a-8a7a-54d44d840758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:04.690891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624bb5a-876d-4b7a-8a7a-54d44d840758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:04.690891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624bb5a-876d-4b7a-8a7a-54d44d840758 admin admin] PATH_INFO: `/v3/users/7bd3e9df98f643948c0d9abf05fea455` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:04.692519 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624bb5a-876d-4b7a-8a7a-54d44d840758 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd3e9df98f643948c0d9abf05fea455)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:04.696366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624bb5a-876d-4b7a-8a7a-54d44d840758 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:04.751486 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2816520b-b7a0-4f44-8460-4305800ba35e None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:04.760231 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2816520b-b7a0-4f44-8460-4305800ba35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:04.840284 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 713/1434] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:04.857966 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816520b-b7a0-4f44-8460-4305800ba35e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:04.859615 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2816520b-b7a0-4f44-8460-4305800ba35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:04.933896 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-8624bb5a-876d-4b7a-8a7a-54d44d840758 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd3e9df98f643948c0d9abf05fea455'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:04.949179 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:04.967545 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-8624bb5a-876d-4b7a-8a7a-54d44d840758 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd3e9df98f643948c0d9abf05fea455'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:04.976162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-8624bb5a-876d-4b7a-8a7a-54d44d840758 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd3e9df98f643948c0d9abf05fea455'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:05.020307 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-8624bb5a-876d-4b7a-8a7a-54d44d840758 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd3e9df98f643948c0d9abf05fea455'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:05.091769 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816520b-b7a0-4f44-8460-4305800ba35e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:05.100353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed894a-3071-4b94-a58c-d353c268e6f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:05.101617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed894a-3071-4b94-a58c-d353c268e6f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:05.103006 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed894a-3071-4b94-a58c-d353c268e6f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:05.103512 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed894a-3071-4b94-a58c-d353c268e6f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:05.103887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-8624bb5a-876d-4b7a-8a7a-54d44d840758 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd3e9df98f643948c0d9abf05fea455'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:05.105883 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-02ed894a-3071-4b94-a58c-d353c268e6f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:05.123287 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 722/1435] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:04 2022] DELETE /identity/v3/users/7bd3e9df98f643948c0d9abf05fea455 => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:05.140204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed894a-3071-4b94-a58c-d353c268e6f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:05.142655 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed894a-3071-4b94-a58c-d353c268e6f5 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:05.159077 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 714/1436] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:06.533711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a97ae5-29a5-4892-99bb-d46d665e97a7 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:06.534278 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a97ae5-29a5-4892-99bb-d46d665e97a7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:06.534644 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a97ae5-29a5-4892-99bb-d46d665e97a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:06.748957 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-34a97ae5-29a5-4892-99bb-d46d665e97a7 None None] MFA Rules not processed for user `280d47ed79f14105a83b18b2cac58b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:06.754281 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-34a97ae5-29a5-4892-99bb-d46d665e97a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:06.967721 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 723/1437] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Oct 17 03:12:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:12:07.116997 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed894a-3071-4b94-a58c-d353c268e6f5 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:07.119331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-02ed894a-3071-4b94-a58c-d353c268e6f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:07.152325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed894a-3071-4b94-a58c-d353c268e6f5 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:07.270674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:07.272587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:07.273242 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:07.274054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9 admin admin] PATH_INFO: `/v3/projects/add0636e6b964fffbf0bb1c395d6f95b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:07.286585 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=add0636e6b964fffbf0bb1c395d6f95b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:07.291045 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:07.338901 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a97ae5-29a5-4892-99bb-d46d665e97a7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:07.340199 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-34a97ae5-29a5-4892-99bb-d46d665e97a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:07.375855 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:07.446940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'add0636e6b964fffbf0bb1c395d6f95b', 'request_id': 'req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:07.470484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'add0636e6b964fffbf0bb1c395d6f95b', 'request_id': 'req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:07.493256 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'add0636e6b964fffbf0bb1c395d6f95b', 'request_id': 'req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:07.535348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9 admin admin] The token cache is being invalidate because project add0636e6b964fffbf0bb1c395d6f95b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:07.536779 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:07.539946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:07.550775 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 715/1438] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:07 2022] DELETE /identity/v3/projects/add0636e6b964fffbf0bb1c395d6f95b => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:12:07.552083 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:07.558267 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:07.579716 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a97ae5-29a5-4892-99bb-d46d665e97a7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:07.720564 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:07.782035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28e4fb-b053-4264-b1bc-e09e297d6dc9 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:07.801963 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af145b3-e3b5-4ce5-be4d-4c1ee8592c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:07.808863 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af145b3-e3b5-4ce5-be4d-4c1ee8592c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:07.809340 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af145b3-e3b5-4ce5-be4d-4c1ee8592c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:07.809843 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af145b3-e3b5-4ce5-be4d-4c1ee8592c74 admin admin] PATH_INFO: `/v3/users/685e4589907346109bf623798d50a6e1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:07.812699 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af145b3-e3b5-4ce5-be4d-4c1ee8592c74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=685e4589907346109bf623798d50a6e1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:07.813560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5c1f5-a6a1-4855-ab04-2a404b377843 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eceda8b3a8ba4233851d0138d8f4d06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:07.817437 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5c1f5-a6a1-4855-ab04-2a404b377843 service swift] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:07.817587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af145b3-e3b5-4ce5-be4d-4c1ee8592c74 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:07.820997 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5c1f5-a6a1-4855-ab04-2a404b377843 service swift] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:07.821339 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5c1f5-a6a1-4855-ab04-2a404b377843 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:07.843958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-87c5c1f5-a6a1-4855-ab04-2a404b377843 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:08.015184 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4af145b3-e3b5-4ce5-be4d-4c1ee8592c74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '685e4589907346109bf623798d50a6e1'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:08.042900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4af145b3-e3b5-4ce5-be4d-4c1ee8592c74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '685e4589907346109bf623798d50a6e1'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:08.046582 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4af145b3-e3b5-4ce5-be4d-4c1ee8592c74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '685e4589907346109bf623798d50a6e1'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:08.070371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4af145b3-e3b5-4ce5-be4d-4c1ee8592c74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '685e4589907346109bf623798d50a6e1'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:08.124414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4af145b3-e3b5-4ce5-be4d-4c1ee8592c74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '685e4589907346109bf623798d50a6e1'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:08.132583 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5c1f5-a6a1-4855-ab04-2a404b377843 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:08.134153 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5c1f5-a6a1-4855-ab04-2a404b377843 service swift] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:08.149335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 716/1439] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:07 2022] DELETE /identity/v3/users/685e4589907346109bf623798d50a6e1 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:08.164624 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 724/1440] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Oct 17 03:12:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 826 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:12:08.797513 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ac32f-889a-4d51-b5f7-f119c909e164 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:08.798048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ac32f-889a-4d51-b5f7-f119c909e164 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:08.798446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ac32f-889a-4d51-b5f7-f119c909e164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:08.894962 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-826ac32f-889a-4d51-b5f7-f119c909e164 None None] MFA Rules not processed for user `1c786544e8fb4e8e8c3a4990a0f36725`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:08.896397 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-826ac32f-889a-4d51-b5f7-f119c909e164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:09.226371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 717/1441] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:09.880673 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5c1f5-a6a1-4855-ab04-2a404b377843 service swift] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:09.882331 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-87c5c1f5-a6a1-4855-ab04-2a404b377843 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:09.915781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5c1f5-a6a1-4855-ab04-2a404b377843 service swift] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:09.993435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:10.065428 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb056312-df30-429b-85f1-7879a0ac11a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:10.066103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb056312-df30-429b-85f1-7879a0ac11a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:10.071062 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb056312-df30-429b-85f1-7879a0ac11a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:10.071724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb056312-df30-429b-85f1-7879a0ac11a7 admin admin] PATH_INFO: `/v3/projects/61b04306c0bf4994b99986bc33a99ba4` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:10.074299 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb056312-df30-429b-85f1-7879a0ac11a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61b04306c0bf4994b99986bc33a99ba4)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:10.076431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb056312-df30-429b-85f1-7879a0ac11a7 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:10.218574 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-bb056312-df30-429b-85f1-7879a0ac11a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61b04306c0bf4994b99986bc33a99ba4', 'request_id': 'req-bb056312-df30-429b-85f1-7879a0ac11a7'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:10.228391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-bb056312-df30-429b-85f1-7879a0ac11a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61b04306c0bf4994b99986bc33a99ba4', 'request_id': 'req-bb056312-df30-429b-85f1-7879a0ac11a7'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:10.237786 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-bb056312-df30-429b-85f1-7879a0ac11a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61b04306c0bf4994b99986bc33a99ba4', 'request_id': 'req-bb056312-df30-429b-85f1-7879a0ac11a7'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:10.290833 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-bb056312-df30-429b-85f1-7879a0ac11a7 admin admin] The token cache is being invalidate because project 61b04306c0bf4994b99986bc33a99ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:10.293759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-bb056312-df30-429b-85f1-7879a0ac11a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:10.295372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-bb056312-df30-429b-85f1-7879a0ac11a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:10.300289 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 725/1442] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:09 2022] DELETE /identity/v3/projects/61b04306c0bf4994b99986bc33a99ba4 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:10.305932 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38e643-ff17-4aea-b1e2-a8e74096e721 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:10.306409 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38e643-ff17-4aea-b1e2-a8e74096e721 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:10.306498 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38e643-ff17-4aea-b1e2-a8e74096e721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:10.364284 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2a38e643-ff17-4aea-b1e2-a8e74096e721 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:10.374714 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2a38e643-ff17-4aea-b1e2-a8e74096e721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:10.466788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d88bf3-5fb9-46a0-b280-9ea21fee8bc3 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:10.467815 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d88bf3-5fb9-46a0-b280-9ea21fee8bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:10.468103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d88bf3-5fb9-46a0-b280-9ea21fee8bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:10.561286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e1d88bf3-5fb9-46a0-b280-9ea21fee8bc3 None None] MFA Rules not processed for user `526933e6ad004493b071f5da5caa09a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:10.575598 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e1d88bf3-5fb9-46a0-b280-9ea21fee8bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:10.711733 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 718/1443] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:10.723985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac715bd-0247-40ce-9c75-86485167a87b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:10.724547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac715bd-0247-40ce-9c75-86485167a87b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:10.725119 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac715bd-0247-40ce-9c75-86485167a87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:10.788823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3ac715bd-0247-40ce-9c75-86485167a87b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:10.793546 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3ac715bd-0247-40ce-9c75-86485167a87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:10.820156 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 719/1444] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:10.823411 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac715bd-0247-40ce-9c75-86485167a87b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:10.861017 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac715bd-0247-40ce-9c75-86485167a87b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:10.878025 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f1641-f1da-48dd-95a0-6b7d8dcbc93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:10.878671 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f1641-f1da-48dd-95a0-6b7d8dcbc93e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:10.882245 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f1641-f1da-48dd-95a0-6b7d8dcbc93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:10.885118 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f1641-f1da-48dd-95a0-6b7d8dcbc93e admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:10.889693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f1641-f1da-48dd-95a0-6b7d8dcbc93e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:12:10.890087 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f1641-f1da-48dd-95a0-6b7d8dcbc93e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:10.904724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 726/1445] 10.222.0.23 () {58 vars in 1015 bytes} [Mon Oct 17 03:12:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:10.905366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f1641-f1da-48dd-95a0-6b7d8dcbc93e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:10.914789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:10.972142 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 720/1446] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:12:10.975447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d88bf3-5fb9-46a0-b280-9ea21fee8bc3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:11.013312 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d88bf3-5fb9-46a0-b280-9ea21fee8bc3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:11.050728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce9a5c-b604-4980-8c0c-bf9cac19dff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:11.051550 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce9a5c-b604-4980-8c0c-bf9cac19dff6 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:11.053345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce9a5c-b604-4980-8c0c-bf9cac19dff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:11.055685 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce9a5c-b604-4980-8c0c-bf9cac19dff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:11.058002 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce9a5c-b604-4980-8c0c-bf9cac19dff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:11.066086 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce9a5c-b604-4980-8c0c-bf9cac19dff6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:11.152130 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f1641-f1da-48dd-95a0-6b7d8dcbc93e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:11.154102 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6e1f1641-f1da-48dd-95a0-6b7d8dcbc93e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:11.193137 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f1641-f1da-48dd-95a0-6b7d8dcbc93e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:11.204234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 727/1447] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:10 2022] POST /identity/v3/projects => generated 385 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:11.205358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce9a5c-b604-4980-8c0c-bf9cac19dff6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:11.207475 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6cce9a5c-b604-4980-8c0c-bf9cac19dff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:11.419178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:11.432789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce9a5c-b604-4980-8c0c-bf9cac19dff6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:11.438972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa334540-ef8d-4866-b443-61776f35a9e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:11.446181 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa334540-ef8d-4866-b443-61776f35a9e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:11.447273 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa334540-ef8d-4866-b443-61776f35a9e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:11.448331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa334540-ef8d-4866-b443-61776f35a9e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:11.453218 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826d180-b126-4a7f-9022-0465f8421abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:11.453887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826d180-b126-4a7f-9022-0465f8421abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:11.454178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826d180-b126-4a7f-9022-0465f8421abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:11.454461 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826d180-b126-4a7f-9022-0465f8421abc admin admin] PATH_INFO: `/v3/projects/e14e704856244b2fa08d1d5513f301fc` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:11.457569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826d180-b126-4a7f-9022-0465f8421abc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e14e704856244b2fa08d1d5513f301fc)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:11.459665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826d180-b126-4a7f-9022-0465f8421abc admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:11.477960 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa334540-ef8d-4866-b443-61776f35a9e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:11.481198 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa334540-ef8d-4866-b443-61776f35a9e9 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:11.509212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 721/1448] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:11.511166 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa334540-ef8d-4866-b443-61776f35a9e9 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:11.548500 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa334540-ef8d-4866-b443-61776f35a9e9 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:11.562476 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2826d180-b126-4a7f-9022-0465f8421abc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e14e704856244b2fa08d1d5513f301fc', 'request_id': 'req-2826d180-b126-4a7f-9022-0465f8421abc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:11.573305 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2826d180-b126-4a7f-9022-0465f8421abc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e14e704856244b2fa08d1d5513f301fc', 'request_id': 'req-2826d180-b126-4a7f-9022-0465f8421abc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:11.590763 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2826d180-b126-4a7f-9022-0465f8421abc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e14e704856244b2fa08d1d5513f301fc', 'request_id': 'req-2826d180-b126-4a7f-9022-0465f8421abc'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:11.615772 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2826d180-b126-4a7f-9022-0465f8421abc admin admin] The token cache is being invalidate because project e14e704856244b2fa08d1d5513f301fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:11.616073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2826d180-b126-4a7f-9022-0465f8421abc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:11.620717 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2826d180-b126-4a7f-9022-0465f8421abc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:11.625954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 728/1449] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:11 2022] DELETE /identity/v3/projects/e14e704856244b2fa08d1d5513f301fc => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:11.627854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533517f6-d7cc-4393-b44e-d8ba7e727cde None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:11.628143 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533517f6-d7cc-4393-b44e-d8ba7e727cde None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:11.628424 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533517f6-d7cc-4393-b44e-d8ba7e727cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:11.677360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-533517f6-d7cc-4393-b44e-d8ba7e727cde None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:11.679195 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-533517f6-d7cc-4393-b44e-d8ba7e727cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:11.688696 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca2f4b-fea7-46fe-a366-7338f09018a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:11.689942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca2f4b-fea7-46fe-a366-7338f09018a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:11.690636 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca2f4b-fea7-46fe-a366-7338f09018a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:11.691522 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca2f4b-fea7-46fe-a366-7338f09018a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:11.693226 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca2f4b-fea7-46fe-a366-7338f09018a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:11.697455 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca2f4b-fea7-46fe-a366-7338f09018a0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:11.714753 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 729/1450] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:12:11.716805 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa37a2-8e96-45f9-bc69-0080ef6c0339 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:11.716942 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa37a2-8e96-45f9-bc69-0080ef6c0339 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:11.717352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa37a2-8e96-45f9-bc69-0080ef6c0339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:11.742943 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 722/1451] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:11 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:11.747239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca2f4b-fea7-46fe-a366-7338f09018a0 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:11.750303 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-eeca2f4b-fea7-46fe-a366-7338f09018a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:11.766260 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-aeaa37a2-8e96-45f9-bc69-0080ef6c0339 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:11.767527 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-aeaa37a2-8e96-45f9-bc69-0080ef6c0339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:11.772002 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca2f4b-fea7-46fe-a366-7338f09018a0 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:11.783548 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b43654-8a05-4984-8ea2-02c7c1c042cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:11.784756 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b43654-8a05-4984-8ea2-02c7c1c042cd admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:11.785789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b43654-8a05-4984-8ea2-02c7c1c042cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:11.787069 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b43654-8a05-4984-8ea2-02c7c1c042cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:11.788788 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b43654-8a05-4984-8ea2-02c7c1c042cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:11.791547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b43654-8a05-4984-8ea2-02c7c1c042cd admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:11.796793 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 730/1452] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:12:11.798072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa37a2-8e96-45f9-bc69-0080ef6c0339 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:11.821493 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 723/1453] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:12:11.825445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b43654-8a05-4984-8ea2-02c7c1c042cd admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:11.831457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa37a2-8e96-45f9-bc69-0080ef6c0339 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:11.847883 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b43654-8a05-4984-8ea2-02c7c1c042cd admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:11.849256 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602df461-9343-43c5-8b6a-745b5091441d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:11.849852 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602df461-9343-43c5-8b6a-745b5091441d admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:11.856125 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602df461-9343-43c5-8b6a-745b5091441d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:11.856710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602df461-9343-43c5-8b6a-745b5091441d admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:11.858649 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602df461-9343-43c5-8b6a-745b5091441d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:12:11.859723 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602df461-9343-43c5-8b6a-745b5091441d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:11.861634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602df461-9343-43c5-8b6a-745b5091441d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:11.867028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade31d7-343c-48b3-97db-1133ab01e951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:11.868320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade31d7-343c-48b3-97db-1133ab01e951 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:11.868655 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade31d7-343c-48b3-97db-1133ab01e951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:11.868865 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade31d7-343c-48b3-97db-1133ab01e951 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:11.869590 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade31d7-343c-48b3-97db-1133ab01e951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:11.871280 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade31d7-343c-48b3-97db-1133ab01e951 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:11.898277 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 724/1454] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:11.899812 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade31d7-343c-48b3-97db-1133ab01e951 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:11.905239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 731/1455] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:12:11.933725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade31d7-343c-48b3-97db-1133ab01e951 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:11.935204 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602df461-9343-43c5-8b6a-745b5091441d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:11.943235 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da19cca-49df-49ef-b265-e4e50c779339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:11.943919 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da19cca-49df-49ef-b265-e4e50c779339 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:11.944221 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da19cca-49df-49ef-b265-e4e50c779339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:11.944691 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da19cca-49df-49ef-b265-e4e50c779339 admin admin] PATH_INFO: `/v3/projects/a659e66f567a4da787b2031c125dc3c0/users/4ff8d8daa9cd498c8b53ce23946247ca/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:11.968624 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602df461-9343-43c5-8b6a-745b5091441d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:11.983316 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061e817-ca38-445d-b1f8-b8dd4ec6927a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:11.983593 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da19cca-49df-49ef-b265-e4e50c779339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a659e66f567a4da787b2031c125dc3c0, user_id=4ff8d8daa9cd498c8b53ce23946247ca, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:11.984491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061e817-ca38-445d-b1f8-b8dd4ec6927a admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:11.985771 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061e817-ca38-445d-b1f8-b8dd4ec6927a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:11.986414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061e817-ca38-445d-b1f8-b8dd4ec6927a admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:11.986984 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da19cca-49df-49ef-b265-e4e50c779339 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:11.987919 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061e817-ca38-445d-b1f8-b8dd4ec6927a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:12.008659 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061e817-ca38-445d-b1f8-b8dd4ec6927a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:12.019786 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 725/1456] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:11 2022] PUT /identity/v3/projects/a659e66f567a4da787b2031c125dc3c0/users/4ff8d8daa9cd498c8b53ce23946247ca/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:12.031358 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948098f-4575-4615-bdc3-498f58d97378 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:12.031651 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948098f-4575-4615-bdc3-498f58d97378 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:12.031962 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948098f-4575-4615-bdc3-498f58d97378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:12.098198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 732/1457] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:11 2022] POST /identity/v3/projects => generated 355 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:12.102353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061e817-ca38-445d-b1f8-b8dd4ec6927a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:12.169315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f948098f-4575-4615-bdc3-498f58d97378 None None] MFA Rules not processed for user `4ff8d8daa9cd498c8b53ce23946247ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:12.170584 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f948098f-4575-4615-bdc3-498f58d97378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:12.261653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061e817-ca38-445d-b1f8-b8dd4ec6927a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:12.280068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f76727-4c89-49fe-9d03-54e467677173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:12.281289 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f76727-4c89-49fe-9d03-54e467677173 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:12.281785 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f76727-4c89-49fe-9d03-54e467677173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:12.282267 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f76727-4c89-49fe-9d03-54e467677173 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:12.284434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f76727-4c89-49fe-9d03-54e467677173 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:12.287107 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f76727-4c89-49fe-9d03-54e467677173 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:12.342481 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 733/1458] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:12 2022] POST /identity/v3/users => generated 278 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:12.343807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:12.347819 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f76727-4c89-49fe-9d03-54e467677173 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:12.368286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f76727-4c89-49fe-9d03-54e467677173 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:12.383580 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d8b19-d10f-476b-b732-28ffa8c20310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:12.384512 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d8b19-d10f-476b-b732-28ffa8c20310 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:12.386510 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d8b19-d10f-476b-b732-28ffa8c20310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:12.390592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d8b19-d10f-476b-b732-28ffa8c20310 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:12.392893 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d8b19-d10f-476b-b732-28ffa8c20310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:12.395994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d8b19-d10f-476b-b732-28ffa8c20310 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:12.430387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 734/1459] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:12:12.434359 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d8b19-d10f-476b-b732-28ffa8c20310 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:12.458301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d8b19-d10f-476b-b732-28ffa8c20310 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:12.465772 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb625c2-6b64-4720-adb8-b284189ae1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:12.466711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb625c2-6b64-4720-adb8-b284189ae1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:12.467361 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb625c2-6b64-4720-adb8-b284189ae1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:12.467862 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb625c2-6b64-4720-adb8-b284189ae1e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:12.468771 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb625c2-6b64-4720-adb8-b284189ae1e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:12.471722 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb625c2-6b64-4720-adb8-b284189ae1e9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:12.505591 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 726/1460] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:12.507327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948098f-4575-4615-bdc3-498f58d97378 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:12.510263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 735/1461] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:12.516425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb625c2-6b64-4720-adb8-b284189ae1e9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:12.542679 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb625c2-6b64-4720-adb8-b284189ae1e9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:12.547763 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948098f-4575-4615-bdc3-498f58d97378 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:12.559913 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fbb4d8-4eb4-483c-8648-b6061157ab98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:12.566294 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a91573-537b-471f-a411-649ca055860c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:12.568058 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fbb4d8-4eb4-483c-8648-b6061157ab98 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:12.568874 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a91573-537b-471f-a411-649ca055860c admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:12.569212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fbb4d8-4eb4-483c-8648-b6061157ab98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:12.570280 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fbb4d8-4eb4-483c-8648-b6061157ab98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:12.570566 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a91573-537b-471f-a411-649ca055860c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:12.570954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a91573-537b-471f-a411-649ca055860c admin admin] PATH_INFO: `/v3/projects/a42a46e166044060b42ff8723e9abfad/users/b06e3c1a580c4c75abfcdbaf4ff18955/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:12.572545 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fbb4d8-4eb4-483c-8648-b6061157ab98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:12.574966 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fbb4d8-4eb4-483c-8648-b6061157ab98 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:12.579932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:12.657613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a91573-537b-471f-a411-649ca055860c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a42a46e166044060b42ff8723e9abfad, user_id=b06e3c1a580c4c75abfcdbaf4ff18955, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:12.675264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a91573-537b-471f-a411-649ca055860c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:12.733440 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 727/1463] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:12 2022] POST /identity/v3/projects => generated 387 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:12.734398 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fbb4d8-4eb4-483c-8648-b6061157ab98 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:12.748223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 736/1463] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:12 2022] PUT /identity/v3/projects/a42a46e166044060b42ff8723e9abfad/users/b06e3c1a580c4c75abfcdbaf4ff18955/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:12.753486 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4ebb1-a784-4f97-bd40-39dba9cfe466 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:12.754213 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4ebb1-a784-4f97-bd40-39dba9cfe466 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:12.754731 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4ebb1-a784-4f97-bd40-39dba9cfe466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:12.831452 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-80f4ebb1-a784-4f97-bd40-39dba9cfe466 None None] MFA Rules not processed for user `b06e3c1a580c4c75abfcdbaf4ff18955`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:12.832992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-80f4ebb1-a784-4f97-bd40-39dba9cfe466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:12.905803 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fbb4d8-4eb4-483c-8648-b6061157ab98 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:12.917998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836771f-49e5-4b69-bb14-3e0ea80b7817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:12.918941 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836771f-49e5-4b69-bb14-3e0ea80b7817 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:12.919411 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836771f-49e5-4b69-bb14-3e0ea80b7817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:12.919866 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836771f-49e5-4b69-bb14-3e0ea80b7817 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:12.920692 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836771f-49e5-4b69-bb14-3e0ea80b7817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:12.923098 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836771f-49e5-4b69-bb14-3e0ea80b7817 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:12.960932 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 728/1464] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:12 2022] POST /identity/v3/users => generated 300 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:12.963110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836771f-49e5-4b69-bb14-3e0ea80b7817 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:12.985349 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836771f-49e5-4b69-bb14-3e0ea80b7817 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:12.987939 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 737/1465] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:12.992900 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4ebb1-a784-4f97-bd40-39dba9cfe466 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:12.999392 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2dbc10-5312-4b50-9ee8-58ea010abe3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:13.000084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dbc10-5312-4b50-9ee8-58ea010abe3e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:13.000486 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dbc10-5312-4b50-9ee8-58ea010abe3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:13.000766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dbc10-5312-4b50-9ee8-58ea010abe3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:13.001425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2dbc10-5312-4b50-9ee8-58ea010abe3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:13.008982 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2dbc10-5312-4b50-9ee8-58ea010abe3e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:13.022036 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4ebb1-a784-4f97-bd40-39dba9cfe466 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:13.029105 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152c2ee-2539-400c-9bb7-f70b9299dbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:13.029995 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152c2ee-2539-400c-9bb7-f70b9299dbf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:13.030202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152c2ee-2539-400c-9bb7-f70b9299dbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:13.030513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152c2ee-2539-400c-9bb7-f70b9299dbf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:13.031211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152c2ee-2539-400c-9bb7-f70b9299dbf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:13.033204 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152c2ee-2539-400c-9bb7-f70b9299dbf1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:13.036676 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 729/1466] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:13.044189 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2dbc10-5312-4b50-9ee8-58ea010abe3e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:13.078805 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2dbc10-5312-4b50-9ee8-58ea010abe3e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:13.098092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e1a88-8c5a-4b25-99df-b9eb9e181580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:13.099659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e1a88-8c5a-4b25-99df-b9eb9e181580 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:13.101942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e1a88-8c5a-4b25-99df-b9eb9e181580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:13.102449 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e1a88-8c5a-4b25-99df-b9eb9e181580 admin admin] PATH_INFO: `/v3/projects/46fbe21b8d8b480e9b7ad5565c36eae7/users/8a9f194f1f854351a1e57416891fe587/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:13.164485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e1a88-8c5a-4b25-99df-b9eb9e181580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46fbe21b8d8b480e9b7ad5565c36eae7, user_id=8a9f194f1f854351a1e57416891fe587, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:13.168453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e1a88-8c5a-4b25-99df-b9eb9e181580 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:13.202642 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 730/1467] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:13 2022] PUT /identity/v3/projects/46fbe21b8d8b480e9b7ad5565c36eae7/users/8a9f194f1f854351a1e57416891fe587/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:12:13.206678 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e1a88-8c5a-4b25-99df-b9eb9e181580 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:13.230764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 738/1468] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:12 2022] POST /identity/v3/projects => generated 357 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:13.233132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152c2ee-2539-400c-9bb7-f70b9299dbf1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:13.368371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e1a88-8c5a-4b25-99df-b9eb9e181580 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:13.375767 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:13.476928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1921a8-5b28-44fc-95ad-5252e5d6243e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:13.477755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1921a8-5b28-44fc-95ad-5252e5d6243e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:13.478010 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1921a8-5b28-44fc-95ad-5252e5d6243e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:13.478309 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1921a8-5b28-44fc-95ad-5252e5d6243e admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:13.479138 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1921a8-5b28-44fc-95ad-5252e5d6243e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:13.481174 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1921a8-5b28-44fc-95ad-5252e5d6243e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:13.497338 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152c2ee-2539-400c-9bb7-f70b9299dbf1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:13.500139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 731/1469] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:13.510075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1921a8-5b28-44fc-95ad-5252e5d6243e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:13.519136 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2678e-0131-48ad-8db1-cca601da9043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:13.519596 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2678e-0131-48ad-8db1-cca601da9043 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:13.519891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2678e-0131-48ad-8db1-cca601da9043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:13.520316 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2678e-0131-48ad-8db1-cca601da9043 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:13.520989 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c2678e-0131-48ad-8db1-cca601da9043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:13.523164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c2678e-0131-48ad-8db1-cca601da9043 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:13.539871 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1921a8-5b28-44fc-95ad-5252e5d6243e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:13.548214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf36dbc-4c92-4900-93a4-4595be5021cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:13.549237 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf36dbc-4c92-4900-93a4-4595be5021cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:13.549344 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf36dbc-4c92-4900-93a4-4595be5021cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:13.549590 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf36dbc-4c92-4900-93a4-4595be5021cb admin admin] PATH_INFO: `/v3/projects/46fbe21b8d8b480e9b7ad5565c36eae7/users/8a9f194f1f854351a1e57416891fe587/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:13.553471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf36dbc-4c92-4900-93a4-4595be5021cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=46fbe21b8d8b480e9b7ad5565c36eae7, user_id=8a9f194f1f854351a1e57416891fe587, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:13.556795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf36dbc-4c92-4900-93a4-4595be5021cb admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:13.607278 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 739/1470] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:13 2022] POST /identity/v3/users => generated 285 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:13.611396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2678e-0131-48ad-8db1-cca601da9043 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:13.644409 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 732/1471] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:13 2022] PUT /identity/v3/projects/46fbe21b8d8b480e9b7ad5565c36eae7/users/8a9f194f1f854351a1e57416891fe587/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:13.648652 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701823d-66be-4770-b26e-2d44b425fcbb None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:13.649081 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701823d-66be-4770-b26e-2d44b425fcbb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:13.651000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2678e-0131-48ad-8db1-cca601da9043 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:13.653604 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701823d-66be-4770-b26e-2d44b425fcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:13.737438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e701823d-66be-4770-b26e-2d44b425fcbb None None] MFA Rules not processed for user `8a9f194f1f854351a1e57416891fe587`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:13.738793 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e701823d-66be-4770-b26e-2d44b425fcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:13.775744 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:13.853086 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2ff50-037a-4c56-9aa7-b845f61f16c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:13.853730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2ff50-037a-4c56-9aa7-b845f61f16c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:13.854090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2ff50-037a-4c56-9aa7-b845f61f16c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:13.854375 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2ff50-037a-4c56-9aa7-b845f61f16c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:13.862603 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2ff50-037a-4c56-9aa7-b845f61f16c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:13.868807 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2ff50-037a-4c56-9aa7-b845f61f16c9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:13.911180 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 740/1472] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:13.912014 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2ff50-037a-4c56-9aa7-b845f61f16c9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:13.939488 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2ff50-037a-4c56-9aa7-b845f61f16c9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:13.967709 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7cae7-499a-4a3c-b078-01331c48f2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:13.968425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7cae7-499a-4a3c-b078-01331c48f2c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:13.968709 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7cae7-499a-4a3c-b078-01331c48f2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:13.968996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7cae7-499a-4a3c-b078-01331c48f2c0 admin admin] PATH_INFO: `/v3/projects/7dc39c0f6af74a218a5fae919d56fc15/users/9293b3cf5635498fac3b25d2ebeba741/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:14.019494 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 733/1473] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:14.025974 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7cae7-499a-4a3c-b078-01331c48f2c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dc39c0f6af74a218a5fae919d56fc15, user_id=9293b3cf5635498fac3b25d2ebeba741, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:14.029016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7cae7-499a-4a3c-b078-01331c48f2c0 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:14.033171 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701823d-66be-4770-b26e-2d44b425fcbb None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:14.035820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e701823d-66be-4770-b26e-2d44b425fcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:14.083144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 741/1474] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:13 2022] PUT /identity/v3/projects/7dc39c0f6af74a218a5fae919d56fc15/users/9293b3cf5635498fac3b25d2ebeba741/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:14.089248 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7cae7-499a-4a3c-b078-01331c48f2c0 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:14.152213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701823d-66be-4770-b26e-2d44b425fcbb None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:14.217702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:14.270673 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7cae7-499a-4a3c-b078-01331c48f2c0 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:14.278079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb39c0ca-46f8-44bd-92c7-cdc52523e896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:14.278079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39c0ca-46f8-44bd-92c7-cdc52523e896 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:14.278079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39c0ca-46f8-44bd-92c7-cdc52523e896 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:14.278079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39c0ca-46f8-44bd-92c7-cdc52523e896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:14.291248 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f1db4-6fe3-4b39-a0e1-699e2489ee95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:14.291945 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f1db4-6fe3-4b39-a0e1-699e2489ee95 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:14.292226 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f1db4-6fe3-4b39-a0e1-699e2489ee95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:14.292502 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f1db4-6fe3-4b39-a0e1-699e2489ee95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:14.293164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f1db4-6fe3-4b39-a0e1-699e2489ee95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:14.294828 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f1db4-6fe3-4b39-a0e1-699e2489ee95 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:14.317228 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 742/1475] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:14.322766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f1db4-6fe3-4b39-a0e1-699e2489ee95 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:14.342095 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f1db4-6fe3-4b39-a0e1-699e2489ee95 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:14.358641 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ffadf-16f2-4e27-9869-db0eb0de0899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:14.359380 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ffadf-16f2-4e27-9869-db0eb0de0899 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:14.359686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ffadf-16f2-4e27-9869-db0eb0de0899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:14.360004 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ffadf-16f2-4e27-9869-db0eb0de0899 admin admin] PATH_INFO: `/v3/projects/7dc39c0f6af74a218a5fae919d56fc15/users/9293b3cf5635498fac3b25d2ebeba741/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:14.368766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ffadf-16f2-4e27-9869-db0eb0de0899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dc39c0f6af74a218a5fae919d56fc15, user_id=9293b3cf5635498fac3b25d2ebeba741, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:14.371972 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ffadf-16f2-4e27-9869-db0eb0de0899 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:14.406308 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 743/1476] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:14 2022] PUT /identity/v3/projects/7dc39c0f6af74a218a5fae919d56fc15/users/9293b3cf5635498fac3b25d2ebeba741/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:12:14.418087 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861402fe-0021-4048-8c0c-61de265cfb48 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:14.418376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861402fe-0021-4048-8c0c-61de265cfb48 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:14.418901 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861402fe-0021-4048-8c0c-61de265cfb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:14.434467 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb39c0ca-46f8-44bd-92c7-cdc52523e896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:14.438419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb39c0ca-46f8-44bd-92c7-cdc52523e896 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:14.461594 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 734/1477] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:14.474516 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-861402fe-0021-4048-8c0c-61de265cfb48 None None] MFA Rules not processed for user `9293b3cf5635498fac3b25d2ebeba741`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:14.477308 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-861402fe-0021-4048-8c0c-61de265cfb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:14.519624 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb39c0ca-46f8-44bd-92c7-cdc52523e896 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:14.535391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb39c0ca-46f8-44bd-92c7-cdc52523e896 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:14.670048 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 744/1478] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:14 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:12:14.682205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00178c-fc7a-4cea-a49b-e601e584d880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:14.682614 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861402fe-0021-4048-8c0c-61de265cfb48 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:14.683832 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00178c-fc7a-4cea-a49b-e601e584d880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:14.684015 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-861402fe-0021-4048-8c0c-61de265cfb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:14.684587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00178c-fc7a-4cea-a49b-e601e584d880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:14.685162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00178c-fc7a-4cea-a49b-e601e584d880 admin admin] PATH_INFO: `/v3/users/4ff8d8daa9cd498c8b53ce23946247ca` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:14.688405 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00178c-fc7a-4cea-a49b-e601e584d880 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ff8d8daa9cd498c8b53ce23946247ca)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:14.693678 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00178c-fc7a-4cea-a49b-e601e584d880 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:14.764079 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:14.813582 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fe00178c-fc7a-4cea-a49b-e601e584d880 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ff8d8daa9cd498c8b53ce23946247ca'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:14.833323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fe00178c-fc7a-4cea-a49b-e601e584d880 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ff8d8daa9cd498c8b53ce23946247ca'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:14.834573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fe00178c-fc7a-4cea-a49b-e601e584d880 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ff8d8daa9cd498c8b53ce23946247ca'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:14.849441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fe00178c-fc7a-4cea-a49b-e601e584d880 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ff8d8daa9cd498c8b53ce23946247ca'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:14.876170 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fe00178c-fc7a-4cea-a49b-e601e584d880 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ff8d8daa9cd498c8b53ce23946247ca'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:14.880583 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861402fe-0021-4048-8c0c-61de265cfb48 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:14.892540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 735/1479] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:14 2022] DELETE /identity/v3/users/4ff8d8daa9cd498c8b53ce23946247ca => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:14.894651 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99e010-4fc9-4b99-87d5-b8c7c1c11424 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:14.894651 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99e010-4fc9-4b99-87d5-b8c7c1c11424 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:14.894912 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99e010-4fc9-4b99-87d5-b8c7c1c11424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:14.927411 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:14.955000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-fb99e010-4fc9-4b99-87d5-b8c7c1c11424 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:14.970118 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fb99e010-4fc9-4b99-87d5-b8c7c1c11424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:15.116480 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 736/1480] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:12:15.120079 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816651a-fff8-40a9-b9a4-c79f0eab74bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:15.121354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816651a-fff8-40a9-b9a4-c79f0eab74bb service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:15.121936 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816651a-fff8-40a9-b9a4-c79f0eab74bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:15.122222 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816651a-fff8-40a9-b9a4-c79f0eab74bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:15.124151 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3816651a-fff8-40a9-b9a4-c79f0eab74bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:15.209337 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99e010-4fc9-4b99-87d5-b8c7c1c11424 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:15.211493 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fb99e010-4fc9-4b99-87d5-b8c7c1c11424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:15.259952 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816651a-fff8-40a9-b9a4-c79f0eab74bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:15.261548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816651a-fff8-40a9-b9a4-c79f0eab74bb service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:15.317343 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 745/1481] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:14 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:12:15.428191 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99e010-4fc9-4b99-87d5-b8c7c1c11424 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:15.434523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2c7ec-2306-4325-bb96-fa9da19589be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:15.435386 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2c7ec-2306-4325-bb96-fa9da19589be service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:15.436124 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2c7ec-2306-4325-bb96-fa9da19589be service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:15.436287 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2c7ec-2306-4325-bb96-fa9da19589be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:15.477146 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2c7ec-2306-4325-bb96-fa9da19589be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:15.478858 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2c7ec-2306-4325-bb96-fa9da19589be service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:15.512706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816651a-fff8-40a9-b9a4-c79f0eab74bb service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:15.514768 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3816651a-fff8-40a9-b9a4-c79f0eab74bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:15.518322 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 737/1482] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:12:15.539231 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816651a-fff8-40a9-b9a4-c79f0eab74bb service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:15.551523 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b4f45-1394-4921-91f3-75c04df895a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:15.552551 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b4f45-1394-4921-91f3-75c04df895a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:15.553151 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b4f45-1394-4921-91f3-75c04df895a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:15.553638 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b4f45-1394-4921-91f3-75c04df895a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:15.555910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5f4b4f45-1394-4921-91f3-75c04df895a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:15.649251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b4f45-1394-4921-91f3-75c04df895a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:15.653483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b4f45-1394-4921-91f3-75c04df895a2 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:15.669834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 746/1483] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:12:17.606041 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b034f3-b139-4578-a5ed-dee177c5f8b6 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:17.606582 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b034f3-b139-4578-a5ed-dee177c5f8b6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:17.607265 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b034f3-b139-4578-a5ed-dee177c5f8b6 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:17.609248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 738/1484] 10.222.0.27 () {54 vars in 921 bytes} [Mon Oct 17 03:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:12:17.660840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342673d-595f-412f-b1dc-0bba3e3508ab None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:17.661481 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342673d-595f-412f-b1dc-0bba3e3508ab None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:17.662026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342673d-595f-412f-b1dc-0bba3e3508ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:17.711827 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e342673d-595f-412f-b1dc-0bba3e3508ab None None] MFA Rules not processed for user `01637fc07ba846aebcb42325bed46216`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:17.713015 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e342673d-595f-412f-b1dc-0bba3e3508ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:17.758568 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:17.918989 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 747/1485] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Oct 17 03:12:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:17.968645 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9002b88-1662-4713-8f5a-ef37e8ae32e6 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:17.969186 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9002b88-1662-4713-8f5a-ef37e8ae32e6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:17.969630 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9002b88-1662-4713-8f5a-ef37e8ae32e6 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:17.971330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 739/1486] 10.222.0.27 () {54 vars in 921 bytes} [Mon Oct 17 03:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:12:18.224398 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c09dc2-aee6-4f7b-acab-f35d14f618f4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:18.224923 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c09dc2-aee6-4f7b-acab-f35d14f618f4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:18.225352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c09dc2-aee6-4f7b-acab-f35d14f618f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:18.283546 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-b0c09dc2-aee6-4f7b-acab-f35d14f618f4 None None] MFA Rules not processed for user `01637fc07ba846aebcb42325bed46216`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:18.286755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b0c09dc2-aee6-4f7b-acab-f35d14f618f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:18.337652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 748/1487] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Oct 17 03:12:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:19.545255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9002b88-1662-4713-8f5a-ef37e8ae32e6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:19.547289 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c9002b88-1662-4713-8f5a-ef37e8ae32e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:19.592115 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9002b88-1662-4713-8f5a-ef37e8ae32e6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:19.600519 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96d866-0370-40a6-8b90-5daf48255c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:19.601650 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96d866-0370-40a6-8b90-5daf48255c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:19.601929 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96d866-0370-40a6-8b90-5daf48255c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:19.602203 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96d866-0370-40a6-8b90-5daf48255c2c admin admin] PATH_INFO: `/v3/projects/a659e66f567a4da787b2031c125dc3c0` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:19.603480 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96d866-0370-40a6-8b90-5daf48255c2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a659e66f567a4da787b2031c125dc3c0)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:19.605348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96d866-0370-40a6-8b90-5daf48255c2c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:19.615084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c09dc2-aee6-4f7b-acab-f35d14f618f4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:19.617320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b0c09dc2-aee6-4f7b-acab-f35d14f618f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:19.643627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c09dc2-aee6-4f7b-acab-f35d14f618f4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:19.651622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe2c093-4abd-4736-bc1f-dc21a988bc47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:19.652413 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2c093-4abd-4736-bc1f-dc21a988bc47 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:19.653018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2c093-4abd-4736-bc1f-dc21a988bc47 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:19.653494 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2c093-4abd-4736-bc1f-dc21a988bc47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:19.670076 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe2c093-4abd-4736-bc1f-dc21a988bc47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:19.672123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe2c093-4abd-4736-bc1f-dc21a988bc47 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:19.673555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ff96d866-0370-40a6-8b90-5daf48255c2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a659e66f567a4da787b2031c125dc3c0', 'request_id': 'req-ff96d866-0370-40a6-8b90-5daf48255c2c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:19.681159 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ff96d866-0370-40a6-8b90-5daf48255c2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a659e66f567a4da787b2031c125dc3c0', 'request_id': 'req-ff96d866-0370-40a6-8b90-5daf48255c2c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:19.683830 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 749/1488] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:19.688290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ff96d866-0370-40a6-8b90-5daf48255c2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a659e66f567a4da787b2031c125dc3c0', 'request_id': 'req-ff96d866-0370-40a6-8b90-5daf48255c2c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:19.688583 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe2c093-4abd-4736-bc1f-dc21a988bc47 service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:19.690763 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5fe2c093-4abd-4736-bc1f-dc21a988bc47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:19.709319 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ff96d866-0370-40a6-8b90-5daf48255c2c admin admin] The token cache is being invalidate because project a659e66f567a4da787b2031c125dc3c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:19.709622 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ff96d866-0370-40a6-8b90-5daf48255c2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:19.711386 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ff96d866-0370-40a6-8b90-5daf48255c2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:19.714725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe2c093-4abd-4736-bc1f-dc21a988bc47 service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:19.716292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 740/1489] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:19 2022] DELETE /identity/v3/projects/a659e66f567a4da787b2031c125dc3c0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:19.717141 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96d866-0370-40a6-8b90-5daf48255c2c admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:19.718629 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ff96d866-0370-40a6-8b90-5daf48255c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:19.767532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:19.821991 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96d866-0370-40a6-8b90-5daf48255c2c admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:19.822562 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333704b1-bbe2-4d4e-960c-687b9d68f61d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:19.823344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333704b1-bbe2-4d4e-960c-687b9d68f61d service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:19.823721 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333704b1-bbe2-4d4e-960c-687b9d68f61d service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:19.823902 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333704b1-bbe2-4d4e-960c-687b9d68f61d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:19.825796 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-333704b1-bbe2-4d4e-960c-687b9d68f61d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:19.827319 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253fd1d-f6c6-4d9d-a83c-3f9fa72ce087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:19.828033 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253fd1d-f6c6-4d9d-a83c-3f9fa72ce087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:19.828394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253fd1d-f6c6-4d9d-a83c-3f9fa72ce087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:19.828742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253fd1d-f6c6-4d9d-a83c-3f9fa72ce087 admin admin] PATH_INFO: `/v3/users/8a9f194f1f854351a1e57416891fe587` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:19.829962 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253fd1d-f6c6-4d9d-a83c-3f9fa72ce087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a9f194f1f854351a1e57416891fe587)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:19.831938 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253fd1d-f6c6-4d9d-a83c-3f9fa72ce087 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:19.839284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333704b1-bbe2-4d4e-960c-687b9d68f61d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:19.840995 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333704b1-bbe2-4d4e-960c-687b9d68f61d service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:19.852688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 750/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:19.901861 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7253fd1d-f6c6-4d9d-a83c-3f9fa72ce087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a9f194f1f854351a1e57416891fe587'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:19.910940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7253fd1d-f6c6-4d9d-a83c-3f9fa72ce087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a9f194f1f854351a1e57416891fe587'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:19.911885 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7253fd1d-f6c6-4d9d-a83c-3f9fa72ce087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a9f194f1f854351a1e57416891fe587'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:19.925953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7253fd1d-f6c6-4d9d-a83c-3f9fa72ce087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a9f194f1f854351a1e57416891fe587'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:19.940587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7253fd1d-f6c6-4d9d-a83c-3f9fa72ce087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a9f194f1f854351a1e57416891fe587'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:19.949053 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 741/1491] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:19 2022] DELETE /identity/v3/users/8a9f194f1f854351a1e57416891fe587 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:20.268712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333704b1-bbe2-4d4e-960c-687b9d68f61d service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:20.285381 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333704b1-bbe2-4d4e-960c-687b9d68f61d service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:20.343524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6ae2c-a5a6-4a4f-bf59-0d9c1a826160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:20.344163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6ae2c-a5a6-4a4f-bf59-0d9c1a826160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:20.344451 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6ae2c-a5a6-4a4f-bf59-0d9c1a826160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:20.344751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6ae2c-a5a6-4a4f-bf59-0d9c1a826160 admin admin] PATH_INFO: `/v3/projects/46fbe21b8d8b480e9b7ad5565c36eae7` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:20.345874 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6ae2c-a5a6-4a4f-bf59-0d9c1a826160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46fbe21b8d8b480e9b7ad5565c36eae7)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:20.347812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6ae2c-a5a6-4a4f-bf59-0d9c1a826160 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:20.372478 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:20.395136 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2fc6ae2c-a5a6-4a4f-bf59-0d9c1a826160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46fbe21b8d8b480e9b7ad5565c36eae7', 'request_id': 'req-2fc6ae2c-a5a6-4a4f-bf59-0d9c1a826160'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:20.401017 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2fc6ae2c-a5a6-4a4f-bf59-0d9c1a826160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46fbe21b8d8b480e9b7ad5565c36eae7', 'request_id': 'req-2fc6ae2c-a5a6-4a4f-bf59-0d9c1a826160'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:20.407345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2fc6ae2c-a5a6-4a4f-bf59-0d9c1a826160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46fbe21b8d8b480e9b7ad5565c36eae7', 'request_id': 'req-2fc6ae2c-a5a6-4a4f-bf59-0d9c1a826160'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:20.420797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2fc6ae2c-a5a6-4a4f-bf59-0d9c1a826160 admin admin] The token cache is being invalidate because project 46fbe21b8d8b480e9b7ad5565c36eae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:20.421088 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2fc6ae2c-a5a6-4a4f-bf59-0d9c1a826160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:20.421767 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2fc6ae2c-a5a6-4a4f-bf59-0d9c1a826160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:20.423270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 751/1492] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:20 2022] DELETE /identity/v3/projects/46fbe21b8d8b480e9b7ad5565c36eae7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:20.428363 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ef1b8-8261-4dfa-a80d-445bd59e8e9b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:20.429027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ef1b8-8261-4dfa-a80d-445bd59e8e9b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:20.429027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ef1b8-8261-4dfa-a80d-445bd59e8e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:20.448509 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a88ef1b8-8261-4dfa-a80d-445bd59e8e9b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:20.449707 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a88ef1b8-8261-4dfa-a80d-445bd59e8e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:20.508171 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 742/1493] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:20.517177 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde1472-fbda-44da-b2c7-d358fc1a0c0b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:20.517469 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde1472-fbda-44da-b2c7-d358fc1a0c0b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:20.517752 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde1472-fbda-44da-b2c7-d358fc1a0c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:20.539201 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2bde1472-fbda-44da-b2c7-d358fc1a0c0b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:20.540448 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2bde1472-fbda-44da-b2c7-d358fc1a0c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:20.548511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 752/1494] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:20.552803 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ef1b8-8261-4dfa-a80d-445bd59e8e9b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:20.562321 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ef1b8-8261-4dfa-a80d-445bd59e8e9b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:20.566846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5febe-1592-42e6-98a6-f710f72d446e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:20.567474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5febe-1592-42e6-98a6-f710f72d446e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:20.567757 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5febe-1592-42e6-98a6-f710f72d446e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:20.568032 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5febe-1592-42e6-98a6-f710f72d446e admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:20.568680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5febe-1592-42e6-98a6-f710f72d446e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:12:20.569011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5febe-1592-42e6-98a6-f710f72d446e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:20.570929 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5febe-1592-42e6-98a6-f710f72d446e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:20.582245 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 743/1495] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:12:20.587271 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde1472-fbda-44da-b2c7-d358fc1a0c0b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:20.596404 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde1472-fbda-44da-b2c7-d358fc1a0c0b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:20.602032 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eed163-7949-4885-b38d-99ca2490ca1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:20.602649 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eed163-7949-4885-b38d-99ca2490ca1d admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:20.602939 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eed163-7949-4885-b38d-99ca2490ca1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:20.603215 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eed163-7949-4885-b38d-99ca2490ca1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:20.603865 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eed163-7949-4885-b38d-99ca2490ca1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:20.605744 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eed163-7949-4885-b38d-99ca2490ca1d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:20.636524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 753/1496] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:20 2022] POST /identity/v3/projects => generated 409 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:20.637595 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5febe-1592-42e6-98a6-f710f72d446e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:20.648531 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5febe-1592-42e6-98a6-f710f72d446e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:20.683486 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:20.715355 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55d36c-0bcb-45d2-8e1e-3a24ced3b22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:20.716065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55d36c-0bcb-45d2-8e1e-3a24ced3b22e admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:20.716411 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55d36c-0bcb-45d2-8e1e-3a24ced3b22e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:20.716748 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55d36c-0bcb-45d2-8e1e-3a24ced3b22e admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:20.717460 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55d36c-0bcb-45d2-8e1e-3a24ced3b22e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:20.720016 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55d36c-0bcb-45d2-8e1e-3a24ced3b22e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:20.744510 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 744/1497] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:20 2022] POST /identity/v3/users => generated 305 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:12:20.748835 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eed163-7949-4885-b38d-99ca2490ca1d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:20.761210 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eed163-7949-4885-b38d-99ca2490ca1d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:20.766492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0d5fa-3a73-4149-8226-f4a92ba5fe96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:20.767116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0d5fa-3a73-4149-8226-f4a92ba5fe96 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:20.767387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0d5fa-3a73-4149-8226-f4a92ba5fe96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:20.767737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0d5fa-3a73-4149-8226-f4a92ba5fe96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:20.768523 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0d5fa-3a73-4149-8226-f4a92ba5fe96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:20.770243 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0d5fa-3a73-4149-8226-f4a92ba5fe96 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:20.782793 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 754/1498] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:12:20.786306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55d36c-0bcb-45d2-8e1e-3a24ced3b22e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:20.798735 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55d36c-0bcb-45d2-8e1e-3a24ced3b22e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:20.804335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14916f8-dbea-4f16-b609-d63da2eb2f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:20.805224 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14916f8-dbea-4f16-b609-d63da2eb2f02 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:20.805583 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14916f8-dbea-4f16-b609-d63da2eb2f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:20.805928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14916f8-dbea-4f16-b609-d63da2eb2f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:20.806653 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14916f8-dbea-4f16-b609-d63da2eb2f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:20.808393 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14916f8-dbea-4f16-b609-d63da2eb2f02 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:20.820526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 745/1499] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:20.824275 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0d5fa-3a73-4149-8226-f4a92ba5fe96 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:20.834241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0d5fa-3a73-4149-8226-f4a92ba5fe96 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:20.838892 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428bb13-543e-4775-b527-e7b1308aa8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:20.839565 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428bb13-543e-4775-b527-e7b1308aa8b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:20.839844 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428bb13-543e-4775-b527-e7b1308aa8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:20.840116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428bb13-543e-4775-b527-e7b1308aa8b9 admin admin] PATH_INFO: `/v3/projects/488c48c6f5ae4d868fc3137a884eb892/users/88a597127b364f6c82aa6f2a1a3919f8/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:20.857082 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6428bb13-543e-4775-b527-e7b1308aa8b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=488c48c6f5ae4d868fc3137a884eb892, user_id=88a597127b364f6c82aa6f2a1a3919f8, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:20.858321 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6428bb13-543e-4775-b527-e7b1308aa8b9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:20.872005 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 755/1500] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:20 2022] PUT /identity/v3/projects/488c48c6f5ae4d868fc3137a884eb892/users/88a597127b364f6c82aa6f2a1a3919f8/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:20.882231 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe95aa-1673-492b-9aab-d8691085c571 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:20.882512 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe95aa-1673-492b-9aab-d8691085c571 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:20.882783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe95aa-1673-492b-9aab-d8691085c571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:20.909669 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4afe95aa-1673-492b-9aab-d8691085c571 None None] MFA Rules not processed for user `88a597127b364f6c82aa6f2a1a3919f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:20.910788 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4afe95aa-1673-492b-9aab-d8691085c571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:20.979669 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 746/1501] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:20 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:20.984115 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428bb13-543e-4775-b527-e7b1308aa8b9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:21.049511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428bb13-543e-4775-b527-e7b1308aa8b9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:21.055286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228905f5-8577-403f-9347-160e16ca0153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:21.055895 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228905f5-8577-403f-9347-160e16ca0153 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:21.056177 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228905f5-8577-403f-9347-160e16ca0153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:21.056482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228905f5-8577-403f-9347-160e16ca0153 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:21.057125 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228905f5-8577-403f-9347-160e16ca0153 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:21.059056 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228905f5-8577-403f-9347-160e16ca0153 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:21.088807 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 756/1502] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:20 2022] POST /identity/v3/projects => generated 407 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:12:21.092670 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe95aa-1673-492b-9aab-d8691085c571 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:21.105581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe95aa-1673-492b-9aab-d8691085c571 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:21.197848 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11b8df-434c-4fb3-91c4-222aeaa468ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:21.198483 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11b8df-434c-4fb3-91c4-222aeaa468ca admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:21.198764 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11b8df-434c-4fb3-91c4-222aeaa468ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:21.199080 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11b8df-434c-4fb3-91c4-222aeaa468ca admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:21.199868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11b8df-434c-4fb3-91c4-222aeaa468ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:21.201986 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11b8df-434c-4fb3-91c4-222aeaa468ca admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:21.236586 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 747/1503] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:21 2022] POST /identity/v3/users => generated 310 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:21.240976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228905f5-8577-403f-9347-160e16ca0153 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:21.252058 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228905f5-8577-403f-9347-160e16ca0153 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:21.257095 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae514398-a66f-49dc-85ee-89f2bd2d2419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:21.257925 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae514398-a66f-49dc-85ee-89f2bd2d2419 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:21.258235 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae514398-a66f-49dc-85ee-89f2bd2d2419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:21.258512 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae514398-a66f-49dc-85ee-89f2bd2d2419 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:21.259206 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae514398-a66f-49dc-85ee-89f2bd2d2419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:21.260937 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae514398-a66f-49dc-85ee-89f2bd2d2419 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:21.273574 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 757/1504] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:21.277370 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11b8df-434c-4fb3-91c4-222aeaa468ca admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:21.288555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11b8df-434c-4fb3-91c4-222aeaa468ca admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:21.295060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08278ced-8d84-4df6-b8b3-61293c20d8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:21.295817 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08278ced-8d84-4df6-b8b3-61293c20d8e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:21.296183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08278ced-8d84-4df6-b8b3-61293c20d8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:21.296541 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08278ced-8d84-4df6-b8b3-61293c20d8e9 admin admin] PATH_INFO: `/v3/projects/852688aba7964b8c82dbf64453198e8a/users/fbbe5e00a2d546dc91db8f4c7d7b1507/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:21.312994 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08278ced-8d84-4df6-b8b3-61293c20d8e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=852688aba7964b8c82dbf64453198e8a, user_id=fbbe5e00a2d546dc91db8f4c7d7b1507, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:21.315708 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08278ced-8d84-4df6-b8b3-61293c20d8e9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:21.330575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 748/1505] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:21 2022] PUT /identity/v3/projects/852688aba7964b8c82dbf64453198e8a/users/fbbe5e00a2d546dc91db8f4c7d7b1507/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:21.334545 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae514398-a66f-49dc-85ee-89f2bd2d2419 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:21.346502 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae514398-a66f-49dc-85ee-89f2bd2d2419 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:21.366712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:21.414363 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0a51c-e181-4b0b-8a49-2b2a502b9512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:21.415014 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0a51c-e181-4b0b-8a49-2b2a502b9512 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:21.415301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0a51c-e181-4b0b-8a49-2b2a502b9512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:21.415603 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0a51c-e181-4b0b-8a49-2b2a502b9512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:21.416246 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0a51c-e181-4b0b-8a49-2b2a502b9512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:21.417883 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0a51c-e181-4b0b-8a49-2b2a502b9512 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:21.427655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 758/1506] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:21.431597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08278ced-8d84-4df6-b8b3-61293c20d8e9 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:21.442049 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08278ced-8d84-4df6-b8b3-61293c20d8e9 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:21.447871 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7fd61-3bce-4b9a-be2b-cf91e46b5bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:21.448588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7fd61-3bce-4b9a-be2b-cf91e46b5bc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:21.448904 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7fd61-3bce-4b9a-be2b-cf91e46b5bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:21.449165 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7fd61-3bce-4b9a-be2b-cf91e46b5bc2 admin admin] PATH_INFO: `/v3/projects/852688aba7964b8c82dbf64453198e8a/users/fbbe5e00a2d546dc91db8f4c7d7b1507/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:21.451105 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7fd61-3bce-4b9a-be2b-cf91e46b5bc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=852688aba7964b8c82dbf64453198e8a, user_id=fbbe5e00a2d546dc91db8f4c7d7b1507, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:21.453765 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7fd61-3bce-4b9a-be2b-cf91e46b5bc2 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:21.456986 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:21.466328 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 749/1507] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:21 2022] PUT /identity/v3/projects/852688aba7964b8c82dbf64453198e8a/users/fbbe5e00a2d546dc91db8f4c7d7b1507/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:21.477620 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11153a26-d267-4107-8160-1da60143258a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:21.477906 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11153a26-d267-4107-8160-1da60143258a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:21.478187 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11153a26-d267-4107-8160-1da60143258a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:21.506575 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-11153a26-d267-4107-8160-1da60143258a None None] MFA Rules not processed for user `fbbe5e00a2d546dc91db8f4c7d7b1507`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:21.508586 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-11153a26-d267-4107-8160-1da60143258a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:21.577287 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 759/1508] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:21 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:21.588746 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7fd61-3bce-4b9a-be2b-cf91e46b5bc2 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:21.590677 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a6e7fd61-3bce-4b9a-be2b-cf91e46b5bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:21.648234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7fd61-3bce-4b9a-be2b-cf91e46b5bc2 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:21.652278 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf4f1f-2d62-479f-8526-4c6fc20e27d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:21.652877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf4f1f-2d62-479f-8526-4c6fc20e27d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:21.653154 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf4f1f-2d62-479f-8526-4c6fc20e27d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:21.653424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf4f1f-2d62-479f-8526-4c6fc20e27d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:21.662040 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf4f1f-2d62-479f-8526-4c6fc20e27d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:21.663555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf4f1f-2d62-479f-8526-4c6fc20e27d1 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:21.672920 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 750/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:21 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:21.902640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11153a26-d267-4107-8160-1da60143258a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:21.916975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11153a26-d267-4107-8160-1da60143258a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:21.922766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21ea6e-3c7a-400f-9ae3-e61c3ac7d9ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:21.923957 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21ea6e-3c7a-400f-9ae3-e61c3ac7d9ec service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:21.925354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21ea6e-3c7a-400f-9ae3-e61c3ac7d9ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:21.925858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21ea6e-3c7a-400f-9ae3-e61c3ac7d9ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:22.088879 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21ea6e-3c7a-400f-9ae3-e61c3ac7d9ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:22.091093 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21ea6e-3c7a-400f-9ae3-e61c3ac7d9ec service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:22.104178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 760/1510] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:21 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:31.668841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=28 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:32.097758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=81 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:33.112925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf4f1f-2d62-479f-8526-4c6fc20e27d1 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:33.268651 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf4f1f-2d62-479f-8526-4c6fc20e27d1 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:33.290066 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b170c-1627-4685-aa17-97fb593a1a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:33.290829 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b170c-1627-4685-aa17-97fb593a1a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:33.293349 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b170c-1627-4685-aa17-97fb593a1a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:33.293744 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b170c-1627-4685-aa17-97fb593a1a22 admin admin] PATH_INFO: `/v3/users/88a597127b364f6c82aa6f2a1a3919f8` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:33.295482 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b170c-1627-4685-aa17-97fb593a1a22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88a597127b364f6c82aa6f2a1a3919f8)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:33.299936 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b170c-1627-4685-aa17-97fb593a1a22 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:33.387350 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3c9b170c-1627-4685-aa17-97fb593a1a22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88a597127b364f6c82aa6f2a1a3919f8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:33.406199 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3c9b170c-1627-4685-aa17-97fb593a1a22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88a597127b364f6c82aa6f2a1a3919f8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:33.408524 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3c9b170c-1627-4685-aa17-97fb593a1a22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88a597127b364f6c82aa6f2a1a3919f8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:33.428542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3c9b170c-1627-4685-aa17-97fb593a1a22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88a597127b364f6c82aa6f2a1a3919f8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:33.456548 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3c9b170c-1627-4685-aa17-97fb593a1a22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88a597127b364f6c82aa6f2a1a3919f8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:33.472878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 751/1511] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:33 2022] DELETE /identity/v3/users/88a597127b364f6c82aa6f2a1a3919f8 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:33.474845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50958f-7ee0-4d50-be5a-3e0bb95a56c5 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:33.474845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50958f-7ee0-4d50-be5a-3e0bb95a56c5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:33.475523 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50958f-7ee0-4d50-be5a-3e0bb95a56c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:33.502853 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0c50958f-7ee0-4d50-be5a-3e0bb95a56c5 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:33.504315 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0c50958f-7ee0-4d50-be5a-3e0bb95a56c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:33.593438 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 761/1512] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:33.614465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b170c-1627-4685-aa17-97fb593a1a22 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:33.618834 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3c9b170c-1627-4685-aa17-97fb593a1a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:33.730191 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b170c-1627-4685-aa17-97fb593a1a22 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:33.734436 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988beb2-7689-43cd-8cac-c0e5dabaaf96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:33.735347 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988beb2-7689-43cd-8cac-c0e5dabaaf96 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:33.735774 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988beb2-7689-43cd-8cac-c0e5dabaaf96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:33.736279 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988beb2-7689-43cd-8cac-c0e5dabaaf96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:33.759007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988beb2-7689-43cd-8cac-c0e5dabaaf96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:33.775369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988beb2-7689-43cd-8cac-c0e5dabaaf96 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:33.786230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 752/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:35.154563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50958f-7ee0-4d50-be5a-3e0bb95a56c5 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:35.156162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0c50958f-7ee0-4d50-be5a-3e0bb95a56c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:35.175876 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50958f-7ee0-4d50-be5a-3e0bb95a56c5 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:35.182600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c061a80-1997-4dd0-ba21-e30cb4bb1fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:35.183349 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c061a80-1997-4dd0-ba21-e30cb4bb1fff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:35.183817 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c061a80-1997-4dd0-ba21-e30cb4bb1fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:35.184098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c061a80-1997-4dd0-ba21-e30cb4bb1fff admin admin] PATH_INFO: `/v3/projects/488c48c6f5ae4d868fc3137a884eb892` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:35.185253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c061a80-1997-4dd0-ba21-e30cb4bb1fff admin admin] RBAC: Authorizing `identity:delete_project(project_id=488c48c6f5ae4d868fc3137a884eb892)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:35.187554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c061a80-1997-4dd0-ba21-e30cb4bb1fff admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:35.267037 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9c061a80-1997-4dd0-ba21-e30cb4bb1fff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '488c48c6f5ae4d868fc3137a884eb892', 'request_id': 'req-9c061a80-1997-4dd0-ba21-e30cb4bb1fff'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:35.289383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9c061a80-1997-4dd0-ba21-e30cb4bb1fff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '488c48c6f5ae4d868fc3137a884eb892', 'request_id': 'req-9c061a80-1997-4dd0-ba21-e30cb4bb1fff'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:35.302558 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9c061a80-1997-4dd0-ba21-e30cb4bb1fff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '488c48c6f5ae4d868fc3137a884eb892', 'request_id': 'req-9c061a80-1997-4dd0-ba21-e30cb4bb1fff'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:35.330904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9c061a80-1997-4dd0-ba21-e30cb4bb1fff admin admin] The token cache is being invalidate because project 488c48c6f5ae4d868fc3137a884eb892 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:35.331427 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9c061a80-1997-4dd0-ba21-e30cb4bb1fff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:35.332530 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9c061a80-1997-4dd0-ba21-e30cb4bb1fff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:35.334277 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 762/1514] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:35 2022] DELETE /identity/v3/projects/488c48c6f5ae4d868fc3137a884eb892 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:35.338646 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988beb2-7689-43cd-8cac-c0e5dabaaf96 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:35.352551 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988beb2-7689-43cd-8cac-c0e5dabaaf96 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:35.390104 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:35.440540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b204d0-43d3-41c0-9a4b-06ffe45063bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:35.441271 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b204d0-43d3-41c0-9a4b-06ffe45063bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:35.441652 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b204d0-43d3-41c0-9a4b-06ffe45063bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:35.441992 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b204d0-43d3-41c0-9a4b-06ffe45063bf admin admin] PATH_INFO: `/v3/users/fbbe5e00a2d546dc91db8f4c7d7b1507` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:35.443315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b204d0-43d3-41c0-9a4b-06ffe45063bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbbe5e00a2d546dc91db8f4c7d7b1507)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:35.446546 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b204d0-43d3-41c0-9a4b-06ffe45063bf admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:35.495265 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-37b204d0-43d3-41c0-9a4b-06ffe45063bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbbe5e00a2d546dc91db8f4c7d7b1507'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:35.502084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-37b204d0-43d3-41c0-9a4b-06ffe45063bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbbe5e00a2d546dc91db8f4c7d7b1507'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:35.502781 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-37b204d0-43d3-41c0-9a4b-06ffe45063bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbbe5e00a2d546dc91db8f4c7d7b1507'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:35.511526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-37b204d0-43d3-41c0-9a4b-06ffe45063bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbbe5e00a2d546dc91db8f4c7d7b1507'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:35.524618 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-37b204d0-43d3-41c0-9a4b-06ffe45063bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbbe5e00a2d546dc91db8f4c7d7b1507'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:35.531248 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 753/1515] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:35 2022] DELETE /identity/v3/users/fbbe5e00a2d546dc91db8f4c7d7b1507 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:35.670622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217f77c-ece3-46d2-98dd-a18e6b455173 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:35.670813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217f77c-ece3-46d2-98dd-a18e6b455173 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:35.671117 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217f77c-ece3-46d2-98dd-a18e6b455173 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:35.672479 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 763/1516] 10.222.0.27 () {54 vars in 921 bytes} [Mon Oct 17 03:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:12:35.685413 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c020c-eaa8-4797-9859-cb9e850927d2 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:35.685787 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c020c-eaa8-4797-9859-cb9e850927d2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:35.686136 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c020c-eaa8-4797-9859-cb9e850927d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:35.729874 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9e7c020c-eaa8-4797-9859-cb9e850927d2 None None] MFA Rules not processed for user `01637fc07ba846aebcb42325bed46216`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:35.731145 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9e7c020c-eaa8-4797-9859-cb9e850927d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:35.903760 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 754/1517] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Oct 17 03:12:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:35.965834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217f77c-ece3-46d2-98dd-a18e6b455173 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:35.967830 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6217f77c-ece3-46d2-98dd-a18e6b455173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:36.004081 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217f77c-ece3-46d2-98dd-a18e6b455173 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:36.012425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf96daa-597e-4cfe-93cb-20d565901ed8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:36.013070 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf96daa-597e-4cfe-93cb-20d565901ed8 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:36.013381 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf96daa-597e-4cfe-93cb-20d565901ed8 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:36.013833 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf96daa-597e-4cfe-93cb-20d565901ed8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:36.021077 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c020c-eaa8-4797-9859-cb9e850927d2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:36.023505 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9e7c020c-eaa8-4797-9859-cb9e850927d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:36.024798 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf96daa-597e-4cfe-93cb-20d565901ed8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:36.026968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf96daa-597e-4cfe-93cb-20d565901ed8 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:36.040414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 764/1518] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:36.097837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:36.151144 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c020c-eaa8-4797-9859-cb9e850927d2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:36.157032 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637afca-feb9-4e7f-adec-12c0f7a3abe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:36.157697 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637afca-feb9-4e7f-adec-12c0f7a3abe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:36.157985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637afca-feb9-4e7f-adec-12c0f7a3abe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:36.158266 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637afca-feb9-4e7f-adec-12c0f7a3abe7 admin admin] PATH_INFO: `/v3/projects/852688aba7964b8c82dbf64453198e8a` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:36.159462 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637afca-feb9-4e7f-adec-12c0f7a3abe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=852688aba7964b8c82dbf64453198e8a)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:36.161587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637afca-feb9-4e7f-adec-12c0f7a3abe7 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:36.228087 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e637afca-feb9-4e7f-adec-12c0f7a3abe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '852688aba7964b8c82dbf64453198e8a', 'request_id': 'req-e637afca-feb9-4e7f-adec-12c0f7a3abe7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:36.238521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e637afca-feb9-4e7f-adec-12c0f7a3abe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '852688aba7964b8c82dbf64453198e8a', 'request_id': 'req-e637afca-feb9-4e7f-adec-12c0f7a3abe7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:36.246246 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e637afca-feb9-4e7f-adec-12c0f7a3abe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '852688aba7964b8c82dbf64453198e8a', 'request_id': 'req-e637afca-feb9-4e7f-adec-12c0f7a3abe7'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:36.265091 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e637afca-feb9-4e7f-adec-12c0f7a3abe7 admin admin] The token cache is being invalidate because project 852688aba7964b8c82dbf64453198e8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:36.265502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e637afca-feb9-4e7f-adec-12c0f7a3abe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:36.266356 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e637afca-feb9-4e7f-adec-12c0f7a3abe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:36.268774 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 755/1519] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:36 2022] DELETE /identity/v3/projects/852688aba7964b8c82dbf64453198e8a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:36.274578 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b6bae-bbf5-4e32-aa8b-54f163805216 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:36.275065 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b6bae-bbf5-4e32-aa8b-54f163805216 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:36.275698 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b6bae-bbf5-4e32-aa8b-54f163805216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:36.306348 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-c65b6bae-bbf5-4e32-aa8b-54f163805216 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:36.307797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c65b6bae-bbf5-4e32-aa8b-54f163805216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:36.395752 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 765/1520] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:36.405059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76629505-e67b-4387-9598-3e1d26fbe694 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:36.405968 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76629505-e67b-4387-9598-3e1d26fbe694 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:36.406275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76629505-e67b-4387-9598-3e1d26fbe694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:36.436130 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-76629505-e67b-4387-9598-3e1d26fbe694 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:36.437555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-76629505-e67b-4387-9598-3e1d26fbe694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:36.466317 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 756/1521] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:12:36.470664 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b6bae-bbf5-4e32-aa8b-54f163805216 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:36.484387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b6bae-bbf5-4e32-aa8b-54f163805216 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:36.491840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ce8ce-ea1c-448b-ace4-de6db6ce41cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:36.492932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ce8ce-ea1c-448b-ace4-de6db6ce41cb admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:36.493907 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ce8ce-ea1c-448b-ace4-de6db6ce41cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:36.494412 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ce8ce-ea1c-448b-ace4-de6db6ce41cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:36.495336 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ce8ce-ea1c-448b-ace4-de6db6ce41cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:12:36.495847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ce8ce-ea1c-448b-ace4-de6db6ce41cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:36.497774 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331ce8ce-ea1c-448b-ace4-de6db6ce41cb admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:36.512237 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:36.523939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76629505-e67b-4387-9598-3e1d26fbe694 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:36.525655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 766/1522] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:12:36.539400 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76629505-e67b-4387-9598-3e1d26fbe694 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:36.544935 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c14763-eeff-4895-b474-4d5c767e0177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:36.545649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c14763-eeff-4895-b474-4d5c767e0177 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:36.546143 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c14763-eeff-4895-b474-4d5c767e0177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:36.546490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c14763-eeff-4895-b474-4d5c767e0177 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:36.547273 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c14763-eeff-4895-b474-4d5c767e0177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:36.549369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c14763-eeff-4895-b474-4d5c767e0177 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:36.578649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 757/1523] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:36 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:12:36.583197 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ce8ce-ea1c-448b-ace4-de6db6ce41cb admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:36.593478 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ce8ce-ea1c-448b-ace4-de6db6ce41cb admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:36.649851 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bac32-7b57-4e9e-a235-d5f9219cf89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:36.650605 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bac32-7b57-4e9e-a235-d5f9219cf89e admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:36.650990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bac32-7b57-4e9e-a235-d5f9219cf89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:36.651263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bac32-7b57-4e9e-a235-d5f9219cf89e admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:36.652217 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9bac32-7b57-4e9e-a235-d5f9219cf89e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:36.654120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9bac32-7b57-4e9e-a235-d5f9219cf89e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:36.675383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 767/1524] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:36 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:12:36.680210 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c14763-eeff-4895-b474-4d5c767e0177 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:36.690621 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c14763-eeff-4895-b474-4d5c767e0177 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:36.695568 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be2381-cb3c-4570-9370-e6b1f05f097e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:36.696168 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be2381-cb3c-4570-9370-e6b1f05f097e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:36.696496 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be2381-cb3c-4570-9370-e6b1f05f097e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:36.696720 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be2381-cb3c-4570-9370-e6b1f05f097e admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:36.697363 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be2381-cb3c-4570-9370-e6b1f05f097e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:36.698997 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be2381-cb3c-4570-9370-e6b1f05f097e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:36.708763 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 758/1525] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:36.712873 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bac32-7b57-4e9e-a235-d5f9219cf89e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:36.722994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bac32-7b57-4e9e-a235-d5f9219cf89e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:36.728371 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5bb12-87bb-475e-84a2-65acb7cee0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:36.728970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5bb12-87bb-475e-84a2-65acb7cee0a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:36.729254 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5bb12-87bb-475e-84a2-65acb7cee0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:36.729532 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5bb12-87bb-475e-84a2-65acb7cee0a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:36.730170 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5bb12-87bb-475e-84a2-65acb7cee0a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:36.731800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5bb12-87bb-475e-84a2-65acb7cee0a6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:36.743141 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 768/1526] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:36.747132 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be2381-cb3c-4570-9370-e6b1f05f097e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:36.757000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be2381-cb3c-4570-9370-e6b1f05f097e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:36.761845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d0e02d-f739-4467-a6c9-23f62830c6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:36.762583 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0e02d-f739-4467-a6c9-23f62830c6d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:36.762894 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0e02d-f739-4467-a6c9-23f62830c6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:36.763172 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0e02d-f739-4467-a6c9-23f62830c6d8 admin admin] PATH_INFO: `/v3/projects/b4516f547fe844cf86fab7ce67359540/users/23ff66444404462bbdd937d0ae44840f/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:36.777648 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d0e02d-f739-4467-a6c9-23f62830c6d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4516f547fe844cf86fab7ce67359540, user_id=23ff66444404462bbdd937d0ae44840f, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:36.780356 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d0e02d-f739-4467-a6c9-23f62830c6d8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:36.793075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 759/1527] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:36 2022] PUT /identity/v3/projects/b4516f547fe844cf86fab7ce67359540/users/23ff66444404462bbdd937d0ae44840f/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:12:36.803256 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fe32b-a78d-49c7-9fb8-ae1f877cd022 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:36.803685 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fe32b-a78d-49c7-9fb8-ae1f877cd022 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:36.804121 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fe32b-a78d-49c7-9fb8-ae1f877cd022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:36.831688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a25fe32b-a78d-49c7-9fb8-ae1f877cd022 None None] MFA Rules not processed for user `23ff66444404462bbdd937d0ae44840f`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:36.832883 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a25fe32b-a78d-49c7-9fb8-ae1f877cd022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:36.894650 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 769/1528] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:36.900108 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d0e02d-f739-4467-a6c9-23f62830c6d8 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:36.964136 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d0e02d-f739-4467-a6c9-23f62830c6d8 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:36.968679 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d03b1-6114-45b1-8a00-1be27ca55139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:36.969318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d03b1-6114-45b1-8a00-1be27ca55139 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:36.969598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d03b1-6114-45b1-8a00-1be27ca55139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:36.969870 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d03b1-6114-45b1-8a00-1be27ca55139 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:36.970514 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d03b1-6114-45b1-8a00-1be27ca55139 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:36.972489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d03b1-6114-45b1-8a00-1be27ca55139 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:36.977752 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:37.005357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 760/1529] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:36 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:37.009827 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fe32b-a78d-49c7-9fb8-ae1f877cd022 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:37.021102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fe32b-a78d-49c7-9fb8-ae1f877cd022 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:37.089969 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4182543-9422-4cf6-a1bf-bf913edd3fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:37.089969 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4182543-9422-4cf6-a1bf-bf913edd3fcd admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:37.090318 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4182543-9422-4cf6-a1bf-bf913edd3fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:37.091255 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4182543-9422-4cf6-a1bf-bf913edd3fcd admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:37.092004 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4182543-9422-4cf6-a1bf-bf913edd3fcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:37.094438 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4182543-9422-4cf6-a1bf-bf913edd3fcd admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:37.107855 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:37.120314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 770/1530] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:37 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:12:37.124615 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d03b1-6114-45b1-8a00-1be27ca55139 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:37.136107 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d03b1-6114-45b1-8a00-1be27ca55139 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:37.148314 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1c4ce-5d59-4c33-a85a-798de51a4758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:37.149196 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1c4ce-5d59-4c33-a85a-798de51a4758 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:37.149591 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1c4ce-5d59-4c33-a85a-798de51a4758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:37.150025 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1c4ce-5d59-4c33-a85a-798de51a4758 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:37.150841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1c4ce-5d59-4c33-a85a-798de51a4758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:37.152825 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1c4ce-5d59-4c33-a85a-798de51a4758 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:37.164956 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 761/1531] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:37.169354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4182543-9422-4cf6-a1bf-bf913edd3fcd admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:37.180431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4182543-9422-4cf6-a1bf-bf913edd3fcd admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:37.185532 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af70f1b-6568-4f18-af0e-fcdcb2d262df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:37.186216 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af70f1b-6568-4f18-af0e-fcdcb2d262df admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:37.186559 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af70f1b-6568-4f18-af0e-fcdcb2d262df admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:37.186852 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af70f1b-6568-4f18-af0e-fcdcb2d262df admin admin] PATH_INFO: `/v3/projects/6f5929f1d40b4415bc9ffe1849003562/users/997cf05578b4482288d42d0e99271a6b/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:37.202949 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af70f1b-6568-4f18-af0e-fcdcb2d262df admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5929f1d40b4415bc9ffe1849003562, user_id=997cf05578b4482288d42d0e99271a6b, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:37.205571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af70f1b-6568-4f18-af0e-fcdcb2d262df admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:37.218167 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 771/1532] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:37 2022] PUT /identity/v3/projects/6f5929f1d40b4415bc9ffe1849003562/users/997cf05578b4482288d42d0e99271a6b/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:37.222044 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1c4ce-5d59-4c33-a85a-798de51a4758 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:37.232589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1c4ce-5d59-4c33-a85a-798de51a4758 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:37.292463 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd43b3a-9d0d-43cb-8009-ae6e483ea311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:37.293122 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd43b3a-9d0d-43cb-8009-ae6e483ea311 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:37.293428 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd43b3a-9d0d-43cb-8009-ae6e483ea311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:37.293754 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd43b3a-9d0d-43cb-8009-ae6e483ea311 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:37.294421 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd43b3a-9d0d-43cb-8009-ae6e483ea311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:37.296058 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd43b3a-9d0d-43cb-8009-ae6e483ea311 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:37.306896 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 762/1533] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:37.311160 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af70f1b-6568-4f18-af0e-fcdcb2d262df admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:37.321997 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af70f1b-6568-4f18-af0e-fcdcb2d262df admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:37.326764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97063394-1f13-4cb5-9c37-3035aeff4cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:37.327461 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97063394-1f13-4cb5-9c37-3035aeff4cc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:37.327746 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97063394-1f13-4cb5-9c37-3035aeff4cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:37.328027 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97063394-1f13-4cb5-9c37-3035aeff4cc2 admin admin] PATH_INFO: `/v3/projects/6f5929f1d40b4415bc9ffe1849003562/users/997cf05578b4482288d42d0e99271a6b/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:37.329984 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97063394-1f13-4cb5-9c37-3035aeff4cc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5929f1d40b4415bc9ffe1849003562, user_id=997cf05578b4482288d42d0e99271a6b, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:37.332663 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97063394-1f13-4cb5-9c37-3035aeff4cc2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:37.342748 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 772/1534] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:37 2022] PUT /identity/v3/projects/6f5929f1d40b4415bc9ffe1849003562/users/997cf05578b4482288d42d0e99271a6b/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:37.354461 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23af04a-d695-46f6-81c7-e144450db270 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:37.354752 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23af04a-d695-46f6-81c7-e144450db270 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:37.355070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23af04a-d695-46f6-81c7-e144450db270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:37.388314 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a23af04a-d695-46f6-81c7-e144450db270 None None] MFA Rules not processed for user `997cf05578b4482288d42d0e99271a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:37.389839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a23af04a-d695-46f6-81c7-e144450db270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:37.459892 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 763/1535] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:37.470988 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97063394-1f13-4cb5-9c37-3035aeff4cc2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:37.472630 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-97063394-1f13-4cb5-9c37-3035aeff4cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:37.682883 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97063394-1f13-4cb5-9c37-3035aeff4cc2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:37.691914 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f874c4-a700-4ad7-a5ae-cca9f7efc5a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:37.692551 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f874c4-a700-4ad7-a5ae-cca9f7efc5a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:37.692810 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f874c4-a700-4ad7-a5ae-cca9f7efc5a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:37.693088 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f874c4-a700-4ad7-a5ae-cca9f7efc5a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:37.702723 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f874c4-a700-4ad7-a5ae-cca9f7efc5a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:37.704195 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f874c4-a700-4ad7-a5ae-cca9f7efc5a7 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:37.714377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 773/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:38.345621 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23af04a-d695-46f6-81c7-e144450db270 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:38.438665 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:38.446403 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23af04a-d695-46f6-81c7-e144450db270 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:38.451521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8db8e5-1d78-40b2-887c-93ed1b17c842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:38.452168 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8db8e5-1d78-40b2-887c-93ed1b17c842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:38.452726 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8db8e5-1d78-40b2-887c-93ed1b17c842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:38.453145 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8db8e5-1d78-40b2-887c-93ed1b17c842 admin admin] PATH_INFO: `/v3/users/23ff66444404462bbdd937d0ae44840f` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:38.454459 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8db8e5-1d78-40b2-887c-93ed1b17c842 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23ff66444404462bbdd937d0ae44840f)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:38.456496 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8db8e5-1d78-40b2-887c-93ed1b17c842 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:38.504344 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6d8db8e5-1d78-40b2-887c-93ed1b17c842 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23ff66444404462bbdd937d0ae44840f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:38.512434 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6d8db8e5-1d78-40b2-887c-93ed1b17c842 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23ff66444404462bbdd937d0ae44840f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:38.513265 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6d8db8e5-1d78-40b2-887c-93ed1b17c842 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23ff66444404462bbdd937d0ae44840f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:38.522339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6d8db8e5-1d78-40b2-887c-93ed1b17c842 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23ff66444404462bbdd937d0ae44840f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:38.539319 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6d8db8e5-1d78-40b2-887c-93ed1b17c842 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23ff66444404462bbdd937d0ae44840f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:38.551982 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 764/1537] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:38 2022] DELETE /identity/v3/users/23ff66444404462bbdd937d0ae44840f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:38.558477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe535d2-6e32-451b-b782-d1d98aae5554 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:38.559059 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe535d2-6e32-451b-b782-d1d98aae5554 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:38.559341 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe535d2-6e32-451b-b782-d1d98aae5554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:38.594893 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-afe535d2-6e32-451b-b782-d1d98aae5554 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:38.596951 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-afe535d2-6e32-451b-b782-d1d98aae5554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:38.858658 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 774/1538] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:12:39.002345 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8db8e5-1d78-40b2-887c-93ed1b17c842 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:39.004125 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6d8db8e5-1d78-40b2-887c-93ed1b17c842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:39.139206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8db8e5-1d78-40b2-887c-93ed1b17c842 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:39.147831 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70afb2ba-e36f-421e-aa3c-99855091b9e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:39.148967 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afb2ba-e36f-421e-aa3c-99855091b9e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:39.149412 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afb2ba-e36f-421e-aa3c-99855091b9e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:39.149700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afb2ba-e36f-421e-aa3c-99855091b9e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:39.169605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70afb2ba-e36f-421e-aa3c-99855091b9e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:39.173326 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70afb2ba-e36f-421e-aa3c-99855091b9e9 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:39.195318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 765/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:39.279946 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe535d2-6e32-451b-b782-d1d98aae5554 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:39.283908 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-afe535d2-6e32-451b-b782-d1d98aae5554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:39.301139 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe535d2-6e32-451b-b782-d1d98aae5554 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:39.311376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4727a25-7e2e-4e51-a9b5-f451451c957a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:39.312043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4727a25-7e2e-4e51-a9b5-f451451c957a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:39.312325 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4727a25-7e2e-4e51-a9b5-f451451c957a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:39.312603 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4727a25-7e2e-4e51-a9b5-f451451c957a admin admin] PATH_INFO: `/v3/users/b06e3c1a580c4c75abfcdbaf4ff18955` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:39.314200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4727a25-7e2e-4e51-a9b5-f451451c957a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b06e3c1a580c4c75abfcdbaf4ff18955)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:39.316169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4727a25-7e2e-4e51-a9b5-f451451c957a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:39.396369 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d4727a25-7e2e-4e51-a9b5-f451451c957a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b06e3c1a580c4c75abfcdbaf4ff18955'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:39.410538 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d4727a25-7e2e-4e51-a9b5-f451451c957a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b06e3c1a580c4c75abfcdbaf4ff18955'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:39.411324 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d4727a25-7e2e-4e51-a9b5-f451451c957a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b06e3c1a580c4c75abfcdbaf4ff18955'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:39.413256 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:39.422213 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d4727a25-7e2e-4e51-a9b5-f451451c957a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b06e3c1a580c4c75abfcdbaf4ff18955'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:39.457048 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d4727a25-7e2e-4e51-a9b5-f451451c957a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b06e3c1a580c4c75abfcdbaf4ff18955'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:39.468566 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 775/1540] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:39 2022] DELETE /identity/v3/users/b06e3c1a580c4c75abfcdbaf4ff18955 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:12:39.474142 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957538d5-e390-423c-8f40-2f7264cecdef None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:39.474530 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957538d5-e390-423c-8f40-2f7264cecdef None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:39.474813 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957538d5-e390-423c-8f40-2f7264cecdef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:39.519317 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-957538d5-e390-423c-8f40-2f7264cecdef None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:39.520669 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-957538d5-e390-423c-8f40-2f7264cecdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:39.696503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 766/1541] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:39.733827 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4727a25-7e2e-4e51-a9b5-f451451c957a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:39.735465 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d4727a25-7e2e-4e51-a9b5-f451451c957a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:39.863493 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4727a25-7e2e-4e51-a9b5-f451451c957a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:39.872500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d8c3a-741e-4c33-8753-0366d7c3aefe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:39.873135 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d8c3a-741e-4c33-8753-0366d7c3aefe service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:39.873499 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d8c3a-741e-4c33-8753-0366d7c3aefe service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:39.873756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d8c3a-741e-4c33-8753-0366d7c3aefe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:39.880418 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957538d5-e390-423c-8f40-2f7264cecdef None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:39.882602 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-957538d5-e390-423c-8f40-2f7264cecdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:39.885899 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d8c3a-741e-4c33-8753-0366d7c3aefe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:39.887528 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d8c3a-741e-4c33-8753-0366d7c3aefe service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:39.902015 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 776/1542] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:39.905003 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957538d5-e390-423c-8f40-2f7264cecdef None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:39.911591 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a4a04-942e-4988-b411-6b43cefc8d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:39.912326 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a4a04-942e-4988-b411-6b43cefc8d07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:39.912762 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a4a04-942e-4988-b411-6b43cefc8d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:39.913782 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a4a04-942e-4988-b411-6b43cefc8d07 admin admin] PATH_INFO: `/v3/projects/b4516f547fe844cf86fab7ce67359540` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:39.915785 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a4a04-942e-4988-b411-6b43cefc8d07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4516f547fe844cf86fab7ce67359540)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:39.917767 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a4a04-942e-4988-b411-6b43cefc8d07 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:39.939294 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:40.009722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6e8a4a04-942e-4988-b411-6b43cefc8d07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4516f547fe844cf86fab7ce67359540', 'request_id': 'req-6e8a4a04-942e-4988-b411-6b43cefc8d07'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:40.020095 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6e8a4a04-942e-4988-b411-6b43cefc8d07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4516f547fe844cf86fab7ce67359540', 'request_id': 'req-6e8a4a04-942e-4988-b411-6b43cefc8d07'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:40.033063 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6e8a4a04-942e-4988-b411-6b43cefc8d07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4516f547fe844cf86fab7ce67359540', 'request_id': 'req-6e8a4a04-942e-4988-b411-6b43cefc8d07'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:40.055330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6e8a4a04-942e-4988-b411-6b43cefc8d07 admin admin] The token cache is being invalidate because project b4516f547fe844cf86fab7ce67359540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:40.055599 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6e8a4a04-942e-4988-b411-6b43cefc8d07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:40.056322 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6e8a4a04-942e-4988-b411-6b43cefc8d07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:40.062167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 767/1543] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:39 2022] DELETE /identity/v3/projects/b4516f547fe844cf86fab7ce67359540 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:40.062467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d8c3a-741e-4c33-8753-0366d7c3aefe service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:40.074619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d8c3a-741e-4c33-8753-0366d7c3aefe service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:40.167824 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a1263-6f65-4c88-9ef5-622c6ec80146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:40.169435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a1263-6f65-4c88-9ef5-622c6ec80146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:40.169732 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a1263-6f65-4c88-9ef5-622c6ec80146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:40.170015 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a1263-6f65-4c88-9ef5-622c6ec80146 admin admin] PATH_INFO: `/v3/users/997cf05578b4482288d42d0e99271a6b` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:40.171753 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a1263-6f65-4c88-9ef5-622c6ec80146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=997cf05578b4482288d42d0e99271a6b)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:40.175134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a1263-6f65-4c88-9ef5-622c6ec80146 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:40.240351 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-953a1263-6f65-4c88-9ef5-622c6ec80146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '997cf05578b4482288d42d0e99271a6b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:40.250271 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-953a1263-6f65-4c88-9ef5-622c6ec80146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '997cf05578b4482288d42d0e99271a6b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:40.251117 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-953a1263-6f65-4c88-9ef5-622c6ec80146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '997cf05578b4482288d42d0e99271a6b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:40.261350 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-953a1263-6f65-4c88-9ef5-622c6ec80146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '997cf05578b4482288d42d0e99271a6b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:40.274726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:40.276432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-953a1263-6f65-4c88-9ef5-622c6ec80146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '997cf05578b4482288d42d0e99271a6b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:40.290635 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 777/1544] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:40 2022] DELETE /identity/v3/users/997cf05578b4482288d42d0e99271a6b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:40.602442 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a4a04-942e-4988-b411-6b43cefc8d07 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:40.606097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6e8a4a04-942e-4988-b411-6b43cefc8d07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:40.644817 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a4a04-942e-4988-b411-6b43cefc8d07 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:40.743336 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:40.746490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:40.746947 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:40.747314 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94 admin admin] PATH_INFO: `/v3/projects/a42a46e166044060b42ff8723e9abfad` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:40.749528 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a42a46e166044060b42ff8723e9abfad)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:40.753436 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:40.911442 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a42a46e166044060b42ff8723e9abfad', 'request_id': 'req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:40.931001 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a42a46e166044060b42ff8723e9abfad', 'request_id': 'req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:40.944782 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a42a46e166044060b42ff8723e9abfad', 'request_id': 'req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:40.973471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94 admin admin] The token cache is being invalidate because project a42a46e166044060b42ff8723e9abfad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:40.973471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:40.973471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:40.985309 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 768/1545] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:40 2022] DELETE /identity/v3/projects/a42a46e166044060b42ff8723e9abfad => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:40.991198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a1263-6f65-4c88-9ef5-622c6ec80146 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:40.992858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-953a1263-6f65-4c88-9ef5-622c6ec80146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:41.018332 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a1263-6f65-4c88-9ef5-622c6ec80146 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:41.041454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:41.043153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:41.129513 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:41.166494 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e5aac-4a21-440f-a37c-2366f3f4bfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:41.173258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e5aac-4a21-440f-a37c-2366f3f4bfe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:41.188465 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e5aac-4a21-440f-a37c-2366f3f4bfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:41.188465 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e5aac-4a21-440f-a37c-2366f3f4bfe9 admin admin] PATH_INFO: `/v3/users/9293b3cf5635498fac3b25d2ebeba741` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:41.191879 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717e5aac-4a21-440f-a37c-2366f3f4bfe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9293b3cf5635498fac3b25d2ebeba741)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:41.195117 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717e5aac-4a21-440f-a37c-2366f3f4bfe9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:41.226405 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde822d-7ebb-4d06-ada8-2b3bf9a24c94 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:41.234383 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f4c14-7660-4d50-ae18-c8d994ad1a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:41.235249 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f4c14-7660-4d50-ae18-c8d994ad1a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:41.235941 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f4c14-7660-4d50-ae18-c8d994ad1a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:41.236957 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f4c14-7660-4d50-ae18-c8d994ad1a4b admin admin] PATH_INFO: `/v3/projects/6f5929f1d40b4415bc9ffe1849003562` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:41.243120 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f4c14-7660-4d50-ae18-c8d994ad1a4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f5929f1d40b4415bc9ffe1849003562)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:41.246344 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f4c14-7660-4d50-ae18-c8d994ad1a4b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:41.291935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-717e5aac-4a21-440f-a37c-2366f3f4bfe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9293b3cf5635498fac3b25d2ebeba741'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:41.308246 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-717e5aac-4a21-440f-a37c-2366f3f4bfe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9293b3cf5635498fac3b25d2ebeba741'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:41.309268 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-717e5aac-4a21-440f-a37c-2366f3f4bfe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9293b3cf5635498fac3b25d2ebeba741'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:41.323888 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-717e5aac-4a21-440f-a37c-2366f3f4bfe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9293b3cf5635498fac3b25d2ebeba741'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:41.326200 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-487f4c14-7660-4d50-ae18-c8d994ad1a4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f5929f1d40b4415bc9ffe1849003562', 'request_id': 'req-487f4c14-7660-4d50-ae18-c8d994ad1a4b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:41.337123 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-487f4c14-7660-4d50-ae18-c8d994ad1a4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f5929f1d40b4415bc9ffe1849003562', 'request_id': 'req-487f4c14-7660-4d50-ae18-c8d994ad1a4b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:41.351888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-487f4c14-7660-4d50-ae18-c8d994ad1a4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f5929f1d40b4415bc9ffe1849003562', 'request_id': 'req-487f4c14-7660-4d50-ae18-c8d994ad1a4b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:41.354571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-717e5aac-4a21-440f-a37c-2366f3f4bfe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9293b3cf5635498fac3b25d2ebeba741'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:41.370726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 778/1546] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:40 2022] DELETE /identity/v3/users/9293b3cf5635498fac3b25d2ebeba741 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:41.392700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-487f4c14-7660-4d50-ae18-c8d994ad1a4b admin admin] The token cache is being invalidate because project 6f5929f1d40b4415bc9ffe1849003562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:41.393013 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-487f4c14-7660-4d50-ae18-c8d994ad1a4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:41.394216 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-487f4c14-7660-4d50-ae18-c8d994ad1a4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:41.403946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 769/1547] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:41 2022] DELETE /identity/v3/projects/6f5929f1d40b4415bc9ffe1849003562 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:12:41.407628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfa8ed-faaf-48e9-9390-ec0dadd5970c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:41.408019 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfa8ed-faaf-48e9-9390-ec0dadd5970c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:41.408396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfa8ed-faaf-48e9-9390-ec0dadd5970c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:41.449108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9ecfa8ed-faaf-48e9-9390-ec0dadd5970c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:41.452622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9ecfa8ed-faaf-48e9-9390-ec0dadd5970c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:41.489048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f4c14-7660-4d50-ae18-c8d994ad1a4b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:41.491774 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-487f4c14-7660-4d50-ae18-c8d994ad1a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:41.564356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 779/1548] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:41.572878 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2ac13-8364-406c-9eca-4dac50afde50 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:41.573840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2ac13-8364-406c-9eca-4dac50afde50 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:41.574106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2ac13-8364-406c-9eca-4dac50afde50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:41.591253 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f4c14-7660-4d50-ae18-c8d994ad1a4b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:41.595752 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4be2ac13-8364-406c-9eca-4dac50afde50 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:41.597046 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4be2ac13-8364-406c-9eca-4dac50afde50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:41.597529 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ae562-48f4-4fff-a1ef-de5df200b078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:41.598360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ae562-48f4-4fff-a1ef-de5df200b078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:41.598847 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ae562-48f4-4fff-a1ef-de5df200b078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:41.599256 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ae562-48f4-4fff-a1ef-de5df200b078 admin admin] PATH_INFO: `/v3/users/7655988a5c374922b937617681e3798d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:41.600839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ae562-48f4-4fff-a1ef-de5df200b078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7655988a5c374922b937617681e3798d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:41.603463 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ae562-48f4-4fff-a1ef-de5df200b078 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:41.614952 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 780/1549] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:12:41.619361 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be2ac13-8364-406c-9eca-4dac50afde50 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:41.634396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be2ac13-8364-406c-9eca-4dac50afde50 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:41.638007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b84cb8-2066-4bd6-88e6-bfb64d748692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:41.638707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b84cb8-2066-4bd6-88e6-bfb64d748692 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:41.639195 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b84cb8-2066-4bd6-88e6-bfb64d748692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:41.639543 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b84cb8-2066-4bd6-88e6-bfb64d748692 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:41.640716 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b84cb8-2066-4bd6-88e6-bfb64d748692 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:12:41.641171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b84cb8-2066-4bd6-88e6-bfb64d748692 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:41.643101 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b84cb8-2066-4bd6-88e6-bfb64d748692 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:41.646379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:41.660412 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 781/1550] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:12:41.666245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b84cb8-2066-4bd6-88e6-bfb64d748692 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:41.680738 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b84cb8-2066-4bd6-88e6-bfb64d748692 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:41.693025 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a2ae8-c64c-4de1-802d-0acea082e710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:41.694751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a2ae8-c64c-4de1-802d-0acea082e710 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:41.695552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a2ae8-c64c-4de1-802d-0acea082e710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:41.696057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a2ae8-c64c-4de1-802d-0acea082e710 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:41.696966 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a2ae8-c64c-4de1-802d-0acea082e710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:41.702300 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a2ae8-c64c-4de1-802d-0acea082e710 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:41.739876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5f8ae562-48f4-4fff-a1ef-de5df200b078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7655988a5c374922b937617681e3798d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:41.752571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 782/1551] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:41 2022] POST /identity/v3/projects => generated 379 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:41.754042 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a2ae8-c64c-4de1-802d-0acea082e710 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:41.754491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5f8ae562-48f4-4fff-a1ef-de5df200b078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7655988a5c374922b937617681e3798d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:41.756206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5f8ae562-48f4-4fff-a1ef-de5df200b078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7655988a5c374922b937617681e3798d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:41.785488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5f8ae562-48f4-4fff-a1ef-de5df200b078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7655988a5c374922b937617681e3798d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:41.788305 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:41.807647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5f8ae562-48f4-4fff-a1ef-de5df200b078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7655988a5c374922b937617681e3798d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:41.823070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 770/1552] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:41 2022] DELETE /identity/v3/users/7655988a5c374922b937617681e3798d => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:41.824678 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c5dc9-055c-47ae-8675-694ee97a83b9 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:41.825088 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c5dc9-055c-47ae-8675-694ee97a83b9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:41.825461 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c5dc9-055c-47ae-8675-694ee97a83b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:41.850827 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a2ae8-c64c-4de1-802d-0acea082e710 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:41.852932 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-0f2c5dc9-055c-47ae-8675-694ee97a83b9 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:41.854255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0f2c5dc9-055c-47ae-8675-694ee97a83b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:41.941142 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe7928-4cc8-44ba-99b9-e6e29c91befd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:41.941812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe7928-4cc8-44ba-99b9-e6e29c91befd admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:41.942151 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe7928-4cc8-44ba-99b9-e6e29c91befd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:41.942256 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 771/1553] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:41.943344 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c5dc9-055c-47ae-8675-694ee97a83b9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:41.947779 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0f2c5dc9-055c-47ae-8675-694ee97a83b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:41.948631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe7928-4cc8-44ba-99b9-e6e29c91befd admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:41.949401 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe7928-4cc8-44ba-99b9-e6e29c91befd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:41.951714 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe7928-4cc8-44ba-99b9-e6e29c91befd admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:41.969544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c5dc9-055c-47ae-8675-694ee97a83b9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:41.979807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ecb27-8a22-4877-829c-44ab9ec4defe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:41.982396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ecb27-8a22-4877-829c-44ab9ec4defe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:41.982665 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ecb27-8a22-4877-829c-44ab9ec4defe admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:41.986979 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 783/1554] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:41 2022] POST /identity/v3/users => generated 290 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:41.988456 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe7928-4cc8-44ba-99b9-e6e29c91befd admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:41.988891 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ecb27-8a22-4877-829c-44ab9ec4defe admin admin] PATH_INFO: `/v3/projects/7dc39c0f6af74a218a5fae919d56fc15` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:41.990632 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ecb27-8a22-4877-829c-44ab9ec4defe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dc39c0f6af74a218a5fae919d56fc15)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:41.991984 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7cbe7928-4cc8-44ba-99b9-e6e29c91befd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:41.993862 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ecb27-8a22-4877-829c-44ab9ec4defe admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:42.058754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe7928-4cc8-44ba-99b9-e6e29c91befd admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:42.074890 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b59ecb27-8a22-4877-829c-44ab9ec4defe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dc39c0f6af74a218a5fae919d56fc15', 'request_id': 'req-b59ecb27-8a22-4877-829c-44ab9ec4defe'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:42.074890 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b59ecb27-8a22-4877-829c-44ab9ec4defe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dc39c0f6af74a218a5fae919d56fc15', 'request_id': 'req-b59ecb27-8a22-4877-829c-44ab9ec4defe'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:42.086372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b59ecb27-8a22-4877-829c-44ab9ec4defe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dc39c0f6af74a218a5fae919d56fc15', 'request_id': 'req-b59ecb27-8a22-4877-829c-44ab9ec4defe'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:42.099535 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:42.105458 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b59ecb27-8a22-4877-829c-44ab9ec4defe admin admin] The token cache is being invalidate because project 7dc39c0f6af74a218a5fae919d56fc15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:42.105684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b59ecb27-8a22-4877-829c-44ab9ec4defe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:42.106489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b59ecb27-8a22-4877-829c-44ab9ec4defe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:42.108958 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 772/1555] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:41 2022] DELETE /identity/v3/projects/7dc39c0f6af74a218a5fae919d56fc15 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:42.109679 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ecb27-8a22-4877-829c-44ab9ec4defe admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:42.111213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b59ecb27-8a22-4877-829c-44ab9ec4defe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:42.131564 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e53b2-6507-4768-b7c2-1f539f4dbfa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:42.132312 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e53b2-6507-4768-b7c2-1f539f4dbfa4 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:42.132613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e53b2-6507-4768-b7c2-1f539f4dbfa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:42.132871 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e53b2-6507-4768-b7c2-1f539f4dbfa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:42.134609 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-135e53b2-6507-4768-b7c2-1f539f4dbfa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:42.181691 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ecb27-8a22-4877-829c-44ab9ec4defe admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:42.186515 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4821107-4cac-4ddd-900c-09b0e75af716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:42.187229 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4821107-4cac-4ddd-900c-09b0e75af716 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:42.187616 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4821107-4cac-4ddd-900c-09b0e75af716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:42.187947 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4821107-4cac-4ddd-900c-09b0e75af716 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:42.188631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4821107-4cac-4ddd-900c-09b0e75af716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:42.190252 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4821107-4cac-4ddd-900c-09b0e75af716 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:42.201356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135e53b2-6507-4768-b7c2-1f539f4dbfa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:42.202915 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135e53b2-6507-4768-b7c2-1f539f4dbfa4 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:42.205228 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:42.225630 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 773/1556] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:42.226360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 784/1557] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:42.226993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9485a-69d3-4c32-be4a-a937521b1072 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:42.227284 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9485a-69d3-4c32-be4a-a937521b1072 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:42.227563 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9485a-69d3-4c32-be4a-a937521b1072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:42.243329 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e53b2-6507-4768-b7c2-1f539f4dbfa4 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:42.250989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9ce9485a-69d3-4c32-be4a-a937521b1072 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:42.253986 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9ce9485a-69d3-4c32-be4a-a937521b1072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:42.258472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e53b2-6507-4768-b7c2-1f539f4dbfa4 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:42.266015 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 774/1558] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:42.266827 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff6e95-6e1a-4e34-8bc9-2b988693b17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:42.268256 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff6e95-6e1a-4e34-8bc9-2b988693b17c admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:42.270655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff6e95-6e1a-4e34-8bc9-2b988693b17c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:42.271007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff6e95-6e1a-4e34-8bc9-2b988693b17c admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:42.271788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff6e95-6e1a-4e34-8bc9-2b988693b17c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:42.281597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a818d0-304d-4cfd-bc74-8b32570234ee None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:42.281597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a818d0-304d-4cfd-bc74-8b32570234ee None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:42.281597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a818d0-304d-4cfd-bc74-8b32570234ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:42.288801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff6e95-6e1a-4e34-8bc9-2b988693b17c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:42.306595 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-53a818d0-304d-4cfd-bc74-8b32570234ee None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:42.307900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-53a818d0-304d-4cfd-bc74-8b32570234ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:42.323071 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 775/1559] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:42.324173 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a818d0-304d-4cfd-bc74-8b32570234ee None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:42.328166 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 785/1560] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:42.329798 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff6e95-6e1a-4e34-8bc9-2b988693b17c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:42.340055 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a818d0-304d-4cfd-bc74-8b32570234ee None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:42.347819 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c78fd1-253d-45e7-9f4f-d03b708b1dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:42.349185 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c78fd1-253d-45e7-9f4f-d03b708b1dfb admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:42.349659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c78fd1-253d-45e7-9f4f-d03b708b1dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:42.349844 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c78fd1-253d-45e7-9f4f-d03b708b1dfb admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:42.351354 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c78fd1-253d-45e7-9f4f-d03b708b1dfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:12:42.351656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c78fd1-253d-45e7-9f4f-d03b708b1dfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:42.353974 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c78fd1-253d-45e7-9f4f-d03b708b1dfb admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:42.361478 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff6e95-6e1a-4e34-8bc9-2b988693b17c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:42.374112 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e583ba4-026c-4164-b55c-497cf7eecabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:42.375081 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e583ba4-026c-4164-b55c-497cf7eecabb admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:42.375614 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 776/1561] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:12:42.376949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c78fd1-253d-45e7-9f4f-d03b708b1dfb admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:42.377973 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e583ba4-026c-4164-b55c-497cf7eecabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:42.378273 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e583ba4-026c-4164-b55c-497cf7eecabb admin admin] PATH_INFO: `/v3/projects/533c2963af4148439c41893651d0e24a/users/3936391fc58f4b0b8c07413446db8abe/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:42.390507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c78fd1-253d-45e7-9f4f-d03b708b1dfb admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:42.398712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fc301-a5c6-4562-bcf5-93ccc7747ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:42.399663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fc301-a5c6-4562-bcf5-93ccc7747ddb admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:42.399952 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fc301-a5c6-4562-bcf5-93ccc7747ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:42.400227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fc301-a5c6-4562-bcf5-93ccc7747ddb admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:42.400877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7fc301-a5c6-4562-bcf5-93ccc7747ddb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:42.401353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e583ba4-026c-4164-b55c-497cf7eecabb admin admin] RBAC: Authorizing `identity:create_grant(project_id=533c2963af4148439c41893651d0e24a, user_id=3936391fc58f4b0b8c07413446db8abe, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:42.403223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7fc301-a5c6-4562-bcf5-93ccc7747ddb admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:42.404650 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e583ba4-026c-4164-b55c-497cf7eecabb admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:42.443917 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 786/1562] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:42 2022] PUT /identity/v3/projects/533c2963af4148439c41893651d0e24a/users/3936391fc58f4b0b8c07413446db8abe/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:42.446701 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cbdd8-6460-4fd4-a9f6-80b97a03121e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:42.447269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cbdd8-6460-4fd4-a9f6-80b97a03121e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:42.448171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cbdd8-6460-4fd4-a9f6-80b97a03121e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:42.464525 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 777/1563] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:42 2022] POST /identity/v3/projects => generated 385 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:42.469222 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fc301-a5c6-4562-bcf5-93ccc7747ddb admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:42.485135 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a08cbdd8-6460-4fd4-a9f6-80b97a03121e None None] MFA Rules not processed for user `3936391fc58f4b0b8c07413446db8abe`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:42.487704 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a08cbdd8-6460-4fd4-a9f6-80b97a03121e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:42.565078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fc301-a5c6-4562-bcf5-93ccc7747ddb admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:42.572472 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def36604-8084-4b4a-9589-a005263ce861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:42.572968 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def36604-8084-4b4a-9589-a005263ce861 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:42.573252 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def36604-8084-4b4a-9589-a005263ce861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:42.573524 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def36604-8084-4b4a-9589-a005263ce861 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:42.574744 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def36604-8084-4b4a-9589-a005263ce861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:42.576841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def36604-8084-4b4a-9589-a005263ce861 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:42.617015 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 778/1564] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:42 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:12:42.630399 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 787/1565] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:42.633673 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def36604-8084-4b4a-9589-a005263ce861 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:42.643950 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08cbdd8-6460-4fd4-a9f6-80b97a03121e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:42.646266 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a08cbdd8-6460-4fd4-a9f6-80b97a03121e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:42.651436 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def36604-8084-4b4a-9589-a005263ce861 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:42.657279 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02422681-80b7-460e-8af9-3b759cab2683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:42.657691 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02422681-80b7-460e-8af9-3b759cab2683 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:42.657993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02422681-80b7-460e-8af9-3b759cab2683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:42.658295 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02422681-80b7-460e-8af9-3b759cab2683 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:42.658985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02422681-80b7-460e-8af9-3b759cab2683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:42.660699 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02422681-80b7-460e-8af9-3b759cab2683 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:42.672878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 779/1566] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:42.676463 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02422681-80b7-460e-8af9-3b759cab2683 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:42.689149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02422681-80b7-460e-8af9-3b759cab2683 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:42.697034 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0b00e-8984-492b-83b5-387946db499e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:42.697837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0b00e-8984-492b-83b5-387946db499e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:42.698218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0b00e-8984-492b-83b5-387946db499e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:42.698593 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0b00e-8984-492b-83b5-387946db499e admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:42.699567 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0b00e-8984-492b-83b5-387946db499e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:42.703074 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0b00e-8984-492b-83b5-387946db499e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:42.716805 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 780/1567] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:12:42.721257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0b00e-8984-492b-83b5-387946db499e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:42.733776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0b00e-8984-492b-83b5-387946db499e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:42.738928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a08ba7-2afb-4250-a20c-062d75e534df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:42.739724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a08ba7-2afb-4250-a20c-062d75e534df admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:42.740057 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a08ba7-2afb-4250-a20c-062d75e534df admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:42.740460 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a08ba7-2afb-4250-a20c-062d75e534df admin admin] PATH_INFO: `/v3/projects/ee1a20f3616745d997c9358ccec1f327/users/98c6e2c0e175444db90947942aefe6a6/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:42.751788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:42.757759 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a08ba7-2afb-4250-a20c-062d75e534df admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee1a20f3616745d997c9358ccec1f327, user_id=98c6e2c0e175444db90947942aefe6a6, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:42.760929 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a08ba7-2afb-4250-a20c-062d75e534df admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:42.798714 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 781/1568] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:42 2022] PUT /identity/v3/projects/ee1a20f3616745d997c9358ccec1f327/users/98c6e2c0e175444db90947942aefe6a6/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:12:42.800808 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab84d00-a302-421e-9563-78e2ebe3829a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:42.801096 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab84d00-a302-421e-9563-78e2ebe3829a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:42.803284 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab84d00-a302-421e-9563-78e2ebe3829a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:42.805760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08cbdd8-6460-4fd4-a9f6-80b97a03121e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:42.873407 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1ab84d00-a302-421e-9563-78e2ebe3829a None None] MFA Rules not processed for user `98c6e2c0e175444db90947942aefe6a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:42.875441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1ab84d00-a302-421e-9563-78e2ebe3829a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:42.944902 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:42.946812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef33b-402e-4af0-8feb-470043f3b39c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:42.949200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef33b-402e-4af0-8feb-470043f3b39c service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:42.949455 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef33b-402e-4af0-8feb-470043f3b39c service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:42.949698 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ef33b-402e-4af0-8feb-470043f3b39c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:43.013538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 782/1569] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:43.014616 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab84d00-a302-421e-9563-78e2ebe3829a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:43.016294 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1ab84d00-a302-421e-9563-78e2ebe3829a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:43.080690 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ef33b-402e-4af0-8feb-470043f3b39c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:43.082257 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ef33b-402e-4af0-8feb-470043f3b39c service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:43.093901 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 788/1570] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:43.095121 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef33b-402e-4af0-8feb-470043f3b39c service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:43.175463 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab84d00-a302-421e-9563-78e2ebe3829a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:43.188433 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21796a8-1e16-4613-99e2-58b437695334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:43.189088 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21796a8-1e16-4613-99e2-58b437695334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:43.189374 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21796a8-1e16-4613-99e2-58b437695334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:43.189644 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21796a8-1e16-4613-99e2-58b437695334 admin admin] PATH_INFO: `/v3/projects/3ee67a643720454c942d616c09ab2f0f` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:43.190307 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ef33b-402e-4af0-8feb-470043f3b39c service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:43.190795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21796a8-1e16-4613-99e2-58b437695334 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ee67a643720454c942d616c09ab2f0f)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:43.192834 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21796a8-1e16-4613-99e2-58b437695334 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:43.196285 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6422794-9a5d-4c7f-b450-381078214731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:43.197188 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6422794-9a5d-4c7f-b450-381078214731 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:43.197618 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6422794-9a5d-4c7f-b450-381078214731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:43.198047 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6422794-9a5d-4c7f-b450-381078214731 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:43.201771 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6422794-9a5d-4c7f-b450-381078214731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:43.204008 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6422794-9a5d-4c7f-b450-381078214731 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:43.243510 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 789/1571] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:43 2022] POST /identity/v3/projects => generated 387 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:43.244914 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6422794-9a5d-4c7f-b450-381078214731 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:43.250277 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b21796a8-1e16-4613-99e2-58b437695334 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ee67a643720454c942d616c09ab2f0f', 'request_id': 'req-b21796a8-1e16-4613-99e2-58b437695334'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:43.258859 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b21796a8-1e16-4613-99e2-58b437695334 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ee67a643720454c942d616c09ab2f0f', 'request_id': 'req-b21796a8-1e16-4613-99e2-58b437695334'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:43.267731 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b21796a8-1e16-4613-99e2-58b437695334 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ee67a643720454c942d616c09ab2f0f', 'request_id': 'req-b21796a8-1e16-4613-99e2-58b437695334'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:43.296241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:43.300861 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b21796a8-1e16-4613-99e2-58b437695334 admin admin] The token cache is being invalidate because project 3ee67a643720454c942d616c09ab2f0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:43.301201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b21796a8-1e16-4613-99e2-58b437695334 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:43.304732 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b21796a8-1e16-4613-99e2-58b437695334 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:43.311465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 783/1572] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:43 2022] DELETE /identity/v3/projects/3ee67a643720454c942d616c09ab2f0f => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:43.312297 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21796a8-1e16-4613-99e2-58b437695334 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:43.316002 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b21796a8-1e16-4613-99e2-58b437695334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:43.345054 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6422794-9a5d-4c7f-b450-381078214731 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:43.351970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ea5e3-3877-4af4-84aa-50104a3aa872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:43.352610 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ea5e3-3877-4af4-84aa-50104a3aa872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:43.352895 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ea5e3-3877-4af4-84aa-50104a3aa872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:43.353170 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ea5e3-3877-4af4-84aa-50104a3aa872 admin admin] PATH_INFO: `/v3/users/3936391fc58f4b0b8c07413446db8abe` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:43.354592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ea5e3-3877-4af4-84aa-50104a3aa872 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3936391fc58f4b0b8c07413446db8abe)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:43.356845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ea5e3-3877-4af4-84aa-50104a3aa872 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:43.405026 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21796a8-1e16-4613-99e2-58b437695334 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:43.410473 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d07ea-0d24-4554-a9a1-a0a05e865af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:43.411106 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d07ea-0d24-4554-a9a1-a0a05e865af7 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:43.411390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d07ea-0d24-4554-a9a1-a0a05e865af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:43.411667 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d07ea-0d24-4554-a9a1-a0a05e865af7 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:43.412330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d07ea-0d24-4554-a9a1-a0a05e865af7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:43.416369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d07ea-0d24-4554-a9a1-a0a05e865af7 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:43.434602 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-102ea5e3-3877-4af4-84aa-50104a3aa872 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3936391fc58f4b0b8c07413446db8abe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:43.446103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-102ea5e3-3877-4af4-84aa-50104a3aa872 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3936391fc58f4b0b8c07413446db8abe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:43.447209 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-102ea5e3-3877-4af4-84aa-50104a3aa872 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3936391fc58f4b0b8c07413446db8abe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:43.447979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 784/1573] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:43 2022] POST /identity/v3/users => generated 294 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:43.449109 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d07ea-0d24-4554-a9a1-a0a05e865af7 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:43.451971 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cf5d07ea-0d24-4554-a9a1-a0a05e865af7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:43.462108 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:43.465896 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-102ea5e3-3877-4af4-84aa-50104a3aa872 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3936391fc58f4b0b8c07413446db8abe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:43.468232 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d07ea-0d24-4554-a9a1-a0a05e865af7 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:43.477527 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c7e98-08ef-42fb-80b3-5a7768689ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:43.478181 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c7e98-08ef-42fb-80b3-5a7768689ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:43.478458 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c7e98-08ef-42fb-80b3-5a7768689ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:43.479024 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c7e98-08ef-42fb-80b3-5a7768689ffc admin admin] PATH_INFO: `/v3/users/5134908908d444adb9994733f2620f7e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:43.480080 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c7e98-08ef-42fb-80b3-5a7768689ffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5134908908d444adb9994733f2620f7e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:43.482047 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c7e98-08ef-42fb-80b3-5a7768689ffc admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:43.485108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-102ea5e3-3877-4af4-84aa-50104a3aa872 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3936391fc58f4b0b8c07413446db8abe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:43.492735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 790/1574] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:43 2022] DELETE /identity/v3/users/3936391fc58f4b0b8c07413446db8abe => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:43.493690 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ea5e3-3877-4af4-84aa-50104a3aa872 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:43.496386 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-102ea5e3-3877-4af4-84aa-50104a3aa872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:43.556427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7b6c7e98-08ef-42fb-80b3-5a7768689ffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5134908908d444adb9994733f2620f7e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:43.581347 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7b6c7e98-08ef-42fb-80b3-5a7768689ffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5134908908d444adb9994733f2620f7e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:43.582452 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7b6c7e98-08ef-42fb-80b3-5a7768689ffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5134908908d444adb9994733f2620f7e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:43.593398 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7b6c7e98-08ef-42fb-80b3-5a7768689ffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5134908908d444adb9994733f2620f7e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:43.612203 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ea5e3-3877-4af4-84aa-50104a3aa872 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:43.618724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12ef24-ff75-4c8b-b34b-1491316efa82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:43.619447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12ef24-ff75-4c8b-b34b-1491316efa82 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:43.619725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12ef24-ff75-4c8b-b34b-1491316efa82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:43.619994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12ef24-ff75-4c8b-b34b-1491316efa82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:43.620638 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12ef24-ff75-4c8b-b34b-1491316efa82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:43.621547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7b6c7e98-08ef-42fb-80b3-5a7768689ffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5134908908d444adb9994733f2620f7e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:43.622238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12ef24-ff75-4c8b-b34b-1491316efa82 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:43.632335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 785/1575] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:43 2022] DELETE /identity/v3/users/5134908908d444adb9994733f2620f7e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:43.637431 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f86c8-11c6-44f2-90e3-9b7c12a89c95 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:43.637940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f86c8-11c6-44f2-90e3-9b7c12a89c95 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:43.638534 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f86c8-11c6-44f2-90e3-9b7c12a89c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:43.640043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 791/1576] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:43.640949 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12ef24-ff75-4c8b-b34b-1491316efa82 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:43.642943 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8e12ef24-ff75-4c8b-b34b-1491316efa82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:43.659524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12ef24-ff75-4c8b-b34b-1491316efa82 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:43.662553 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b02f86c8-11c6-44f2-90e3-9b7c12a89c95 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:43.665820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b02f86c8-11c6-44f2-90e3-9b7c12a89c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:43.722485 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:43.742845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3853c6-6a92-444a-98b2-ace595bd5130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:43.743619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3853c6-6a92-444a-98b2-ace595bd5130 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:43.743903 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3853c6-6a92-444a-98b2-ace595bd5130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:43.744177 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3853c6-6a92-444a-98b2-ace595bd5130 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:43.744900 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3853c6-6a92-444a-98b2-ace595bd5130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:43.746594 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3853c6-6a92-444a-98b2-ace595bd5130 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:43.754889 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 786/1577] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:43.760077 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 792/1578] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:43.764444 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f86c8-11c6-44f2-90e3-9b7c12a89c95 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:43.765390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3853c6-6a92-444a-98b2-ace595bd5130 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:43.768045 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1c3853c6-6a92-444a-98b2-ace595bd5130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:43.775714 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f86c8-11c6-44f2-90e3-9b7c12a89c95 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:43.781676 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7633f-2a7e-4465-bff5-ea7079c5aa82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:43.782400 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7633f-2a7e-4465-bff5-ea7079c5aa82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:43.782680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7633f-2a7e-4465-bff5-ea7079c5aa82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:43.783024 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7633f-2a7e-4465-bff5-ea7079c5aa82 admin admin] PATH_INFO: `/v3/projects/cc017af0f3ad409b87a9555dcddf735e/users/4d69dc883be94945a24ae90fca730595/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:43.800004 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7633f-2a7e-4465-bff5-ea7079c5aa82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc017af0f3ad409b87a9555dcddf735e, user_id=4d69dc883be94945a24ae90fca730595, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:43.802665 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7633f-2a7e-4465-bff5-ea7079c5aa82 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:43.830994 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 787/1579] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:43 2022] PUT /identity/v3/projects/cc017af0f3ad409b87a9555dcddf735e/users/4d69dc883be94945a24ae90fca730595/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:43.841591 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45565fc8-b23b-4a19-94bf-2bddf63e5550 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:43.841883 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45565fc8-b23b-4a19-94bf-2bddf63e5550 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:43.842163 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45565fc8-b23b-4a19-94bf-2bddf63e5550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:43.857945 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3853c6-6a92-444a-98b2-ace595bd5130 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:43.880572 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-45565fc8-b23b-4a19-94bf-2bddf63e5550 None None] MFA Rules not processed for user `4d69dc883be94945a24ae90fca730595`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:43.882230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-45565fc8-b23b-4a19-94bf-2bddf63e5550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:43.915964 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6674f2-b7c1-4d06-a833-26b88f15fafd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:43.917462 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6674f2-b7c1-4d06-a833-26b88f15fafd service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:43.917750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6674f2-b7c1-4d06-a833-26b88f15fafd service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:43.918025 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6674f2-b7c1-4d06-a833-26b88f15fafd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:43.994262 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6674f2-b7c1-4d06-a833-26b88f15fafd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:43.995757 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6674f2-b7c1-4d06-a833-26b88f15fafd service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:44.007747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 793/1580] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:12:44.056256 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:44.114622 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 788/1581] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:44.116467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6674f2-b7c1-4d06-a833-26b88f15fafd service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:44.130315 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6674f2-b7c1-4d06-a833-26b88f15fafd service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:44.135940 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970b5dd-578f-44d8-9635-d4fb07bc485b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:44.136628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970b5dd-578f-44d8-9635-d4fb07bc485b admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:44.137017 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970b5dd-578f-44d8-9635-d4fb07bc485b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:44.137362 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970b5dd-578f-44d8-9635-d4fb07bc485b admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:44.138112 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970b5dd-578f-44d8-9635-d4fb07bc485b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:44.140205 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970b5dd-578f-44d8-9635-d4fb07bc485b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:44.172183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 794/1582] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:44 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:44.176194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45565fc8-b23b-4a19-94bf-2bddf63e5550 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:44.193281 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45565fc8-b23b-4a19-94bf-2bddf63e5550 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:44.265458 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7f212-a576-4fd7-afe3-889942a9aa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:44.266246 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7f212-a576-4fd7-afe3-889942a9aa61 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:44.266546 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7f212-a576-4fd7-afe3-889942a9aa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:44.266806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7f212-a576-4fd7-afe3-889942a9aa61 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:44.267490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7f212-a576-4fd7-afe3-889942a9aa61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:44.269480 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7f212-a576-4fd7-afe3-889942a9aa61 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:44.299380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 789/1583] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:44 2022] POST /identity/v3/users => generated 300 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:44.302008 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970b5dd-578f-44d8-9635-d4fb07bc485b admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:44.313854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970b5dd-578f-44d8-9635-d4fb07bc485b admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:44.318953 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4a8f4-8c7e-4fd4-8bfe-694418abf07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:44.319921 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4a8f4-8c7e-4fd4-8bfe-694418abf07e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:44.320064 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4a8f4-8c7e-4fd4-8bfe-694418abf07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:44.320474 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4a8f4-8c7e-4fd4-8bfe-694418abf07e admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:44.321142 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4a8f4-8c7e-4fd4-8bfe-694418abf07e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:44.322905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4a8f4-8c7e-4fd4-8bfe-694418abf07e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:44.335919 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 795/1584] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:12:44.340608 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7f212-a576-4fd7-afe3-889942a9aa61 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:44.353089 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7f212-a576-4fd7-afe3-889942a9aa61 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:44.358941 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7ebd7-d5b8-4f3f-9494-2eef99d31456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:44.360908 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4a8f4-8c7e-4fd4-8bfe-694418abf07e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:44.363021 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-39d4a8f4-8c7e-4fd4-8bfe-694418abf07e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:44.367895 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7ebd7-d5b8-4f3f-9494-2eef99d31456 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:44.372464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7ebd7-d5b8-4f3f-9494-2eef99d31456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:44.372757 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7ebd7-d5b8-4f3f-9494-2eef99d31456 admin admin] PATH_INFO: `/v3/projects/abddd56fbc0244ba954215e74af097ce/users/13377b7ace454154bc8bd06eb75b6198/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:44.380444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4a8f4-8c7e-4fd4-8bfe-694418abf07e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:44.387023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc46c3a0-0728-4ac7-a482-0022dd7542f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:44.387699 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46c3a0-0728-4ac7-a482-0022dd7542f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:44.387972 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46c3a0-0728-4ac7-a482-0022dd7542f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:44.388245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc46c3a0-0728-4ac7-a482-0022dd7542f6 admin admin] PATH_INFO: `/v3/projects/533c2963af4148439c41893651d0e24a` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:44.389735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc46c3a0-0728-4ac7-a482-0022dd7542f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=533c2963af4148439c41893651d0e24a)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:44.393186 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc46c3a0-0728-4ac7-a482-0022dd7542f6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:44.393576 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7ebd7-d5b8-4f3f-9494-2eef99d31456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abddd56fbc0244ba954215e74af097ce, user_id=13377b7ace454154bc8bd06eb75b6198, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:44.396821 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7ebd7-d5b8-4f3f-9494-2eef99d31456 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:44.402968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:44.419035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 790/1585] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:44 2022] PUT /identity/v3/projects/abddd56fbc0244ba954215e74af097ce/users/13377b7ace454154bc8bd06eb75b6198/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:44.423018 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7ebd7-d5b8-4f3f-9494-2eef99d31456 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:44.478895 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dc46c3a0-0728-4ac7-a482-0022dd7542f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '533c2963af4148439c41893651d0e24a', 'request_id': 'req-dc46c3a0-0728-4ac7-a482-0022dd7542f6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:44.489795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dc46c3a0-0728-4ac7-a482-0022dd7542f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '533c2963af4148439c41893651d0e24a', 'request_id': 'req-dc46c3a0-0728-4ac7-a482-0022dd7542f6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:44.497744 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dc46c3a0-0728-4ac7-a482-0022dd7542f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '533c2963af4148439c41893651d0e24a', 'request_id': 'req-dc46c3a0-0728-4ac7-a482-0022dd7542f6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:44.523690 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dc46c3a0-0728-4ac7-a482-0022dd7542f6 admin admin] The token cache is being invalidate because project 533c2963af4148439c41893651d0e24a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:44.523995 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dc46c3a0-0728-4ac7-a482-0022dd7542f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:44.526449 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dc46c3a0-0728-4ac7-a482-0022dd7542f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:44.532061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 796/1586] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:44 2022] DELETE /identity/v3/projects/533c2963af4148439c41893651d0e24a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:44.534323 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d221f5-25a6-4f5e-abd0-2438eeac1e0b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:44.534825 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d221f5-25a6-4f5e-abd0-2438eeac1e0b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:44.535351 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d221f5-25a6-4f5e-abd0-2438eeac1e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:44.538561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7ebd7-d5b8-4f3f-9494-2eef99d31456 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:44.561110 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-b9d221f5-25a6-4f5e-abd0-2438eeac1e0b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:44.562677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b9d221f5-25a6-4f5e-abd0-2438eeac1e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:44.640426 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:44.659303 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca65fa-1146-4ea9-8f56-7b176123f0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:44.659934 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca65fa-1146-4ea9-8f56-7b176123f0ad admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:44.660217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca65fa-1146-4ea9-8f56-7b176123f0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:44.660488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca65fa-1146-4ea9-8f56-7b176123f0ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:44.661280 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca65fa-1146-4ea9-8f56-7b176123f0ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:44.664704 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca65fa-1146-4ea9-8f56-7b176123f0ad admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:44.682326 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 797/1587] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:12:44.684649 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc57b1-cf80-45d5-956f-5a09135010e4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:44.684948 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc57b1-cf80-45d5-956f-5a09135010e4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:44.685224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcc57b1-cf80-45d5-956f-5a09135010e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:44.691068 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 791/1588] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:44.695555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca65fa-1146-4ea9-8f56-7b176123f0ad admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:44.697957 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-25ca65fa-1146-4ea9-8f56-7b176123f0ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:44.725635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca65fa-1146-4ea9-8f56-7b176123f0ad admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:44.731942 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0dcc57b1-cf80-45d5-956f-5a09135010e4 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:44.732668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295833fb-5170-4041-b51a-ff30db736c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:44.733141 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0dcc57b1-cf80-45d5-956f-5a09135010e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:44.733950 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295833fb-5170-4041-b51a-ff30db736c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:44.734435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295833fb-5170-4041-b51a-ff30db736c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:44.735354 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295833fb-5170-4041-b51a-ff30db736c86 admin admin] PATH_INFO: `/v3/projects/abddd56fbc0244ba954215e74af097ce/users/13377b7ace454154bc8bd06eb75b6198/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:44.739250 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295833fb-5170-4041-b51a-ff30db736c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abddd56fbc0244ba954215e74af097ce, user_id=13377b7ace454154bc8bd06eb75b6198, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:44.742188 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 798/1589] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:44.743917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295833fb-5170-4041-b51a-ff30db736c86 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:44.746510 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc57b1-cf80-45d5-956f-5a09135010e4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:44.757117 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcc57b1-cf80-45d5-956f-5a09135010e4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:44.765787 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 792/1590] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:44 2022] PUT /identity/v3/projects/abddd56fbc0244ba954215e74af097ce/users/13377b7ace454154bc8bd06eb75b6198/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:44.780247 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e8677-75f4-40b8-ac74-d06f66417c4a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:44.780541 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e8677-75f4-40b8-ac74-d06f66417c4a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:44.780821 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e8677-75f4-40b8-ac74-d06f66417c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:44.838058 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2b6e8677-75f4-40b8-ac74-d06f66417c4a None None] MFA Rules not processed for user `13377b7ace454154bc8bd06eb75b6198`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:44.840659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2b6e8677-75f4-40b8-ac74-d06f66417c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:44.867614 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b06b6-9d56-4a71-8ec8-7a9d602fe8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:44.868647 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b06b6-9d56-4a71-8ec8-7a9d602fe8da admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:44.869387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b06b6-9d56-4a71-8ec8-7a9d602fe8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:44.869884 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b06b6-9d56-4a71-8ec8-7a9d602fe8da admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:44.870801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b06b6-9d56-4a71-8ec8-7a9d602fe8da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:12:44.871449 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b06b6-9d56-4a71-8ec8-7a9d602fe8da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:44.874059 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b06b6-9d56-4a71-8ec8-7a9d602fe8da admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:44.895227 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 799/1591] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:12:44.899906 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b06b6-9d56-4a71-8ec8-7a9d602fe8da admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:44.926399 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b06b6-9d56-4a71-8ec8-7a9d602fe8da admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:44.934856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19e90a-ce40-4847-b685-8c92081e08bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:44.935842 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19e90a-ce40-4847-b685-8c92081e08bd admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:44.936883 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19e90a-ce40-4847-b685-8c92081e08bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:44.937609 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19e90a-ce40-4847-b685-8c92081e08bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:44.938556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19e90a-ce40-4847-b685-8c92081e08bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:44.941717 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19e90a-ce40-4847-b685-8c92081e08bd admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:44.942080 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 793/1592] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:44.960213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e8677-75f4-40b8-ac74-d06f66417c4a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:44.962406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2b6e8677-75f4-40b8-ac74-d06f66417c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:44.969997 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:44.979553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 800/1593] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:44 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:44.983898 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19e90a-ce40-4847-b685-8c92081e08bd admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:45.036119 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e8677-75f4-40b8-ac74-d06f66417c4a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:45.070970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19e90a-ce40-4847-b685-8c92081e08bd admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:45.077156 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a3c8b-d0af-4148-8702-3b5d044ad428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:45.077785 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a3c8b-d0af-4148-8702-3b5d044ad428 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:45.078119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a3c8b-d0af-4148-8702-3b5d044ad428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:45.078397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a3c8b-d0af-4148-8702-3b5d044ad428 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:45.079127 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a3c8b-d0af-4148-8702-3b5d044ad428 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:45.084734 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a3c8b-d0af-4148-8702-3b5d044ad428 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:45.117151 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf088b5-dd08-4a62-ad82-573310de3577 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:45.117966 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf088b5-dd08-4a62-ad82-573310de3577 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:45.118433 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf088b5-dd08-4a62-ad82-573310de3577 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:45.118805 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf088b5-dd08-4a62-ad82-573310de3577 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:45.124509 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 801/1594] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:44 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:45.125514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a3c8b-d0af-4148-8702-3b5d044ad428 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:45.128983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-292a3c8b-d0af-4148-8702-3b5d044ad428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:45.143789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a3c8b-d0af-4148-8702-3b5d044ad428 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:45.156631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff24bc-8ff0-431e-80d6-76a64778838f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:45.157508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff24bc-8ff0-431e-80d6-76a64778838f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:45.158309 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff24bc-8ff0-431e-80d6-76a64778838f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:45.158803 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff24bc-8ff0-431e-80d6-76a64778838f admin admin] PATH_INFO: `/v3/projects/01cff7d6b3784769931015eba05331b5` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:45.160360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff24bc-8ff0-431e-80d6-76a64778838f admin admin] RBAC: Authorizing `identity:delete_project(project_id=01cff7d6b3784769931015eba05331b5)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:45.164093 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff24bc-8ff0-431e-80d6-76a64778838f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:45.170064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:45.205459 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf088b5-dd08-4a62-ad82-573310de3577 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:45.207338 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf088b5-dd08-4a62-ad82-573310de3577 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:45.219528 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-28ff24bc-8ff0-431e-80d6-76a64778838f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01cff7d6b3784769931015eba05331b5', 'request_id': 'req-28ff24bc-8ff0-431e-80d6-76a64778838f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:45.224947 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 794/1595] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:44 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:12:45.226016 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf088b5-dd08-4a62-ad82-573310de3577 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:45.232754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-28ff24bc-8ff0-431e-80d6-76a64778838f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01cff7d6b3784769931015eba05331b5', 'request_id': 'req-28ff24bc-8ff0-431e-80d6-76a64778838f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:45.237489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf088b5-dd08-4a62-ad82-573310de3577 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:45.244218 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-28ff24bc-8ff0-431e-80d6-76a64778838f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01cff7d6b3784769931015eba05331b5', 'request_id': 'req-28ff24bc-8ff0-431e-80d6-76a64778838f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:45.273062 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-28ff24bc-8ff0-431e-80d6-76a64778838f admin admin] The token cache is being invalidate because project 01cff7d6b3784769931015eba05331b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:45.273711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-28ff24bc-8ff0-431e-80d6-76a64778838f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:45.274893 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-28ff24bc-8ff0-431e-80d6-76a64778838f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:45.277453 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 802/1596] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:45 2022] DELETE /identity/v3/projects/01cff7d6b3784769931015eba05331b5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:45.289795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61905ccb-bfd7-4dc1-86b6-aaab04c691ad None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:45.290074 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61905ccb-bfd7-4dc1-86b6-aaab04c691ad None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:45.290461 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61905ccb-bfd7-4dc1-86b6-aaab04c691ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:45.314782 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-61905ccb-bfd7-4dc1-86b6-aaab04c691ad None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:45.316069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-61905ccb-bfd7-4dc1-86b6-aaab04c691ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:45.324936 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f88e8-ec03-4edc-a922-283078d8afd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:45.325555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f88e8-ec03-4edc-a922-283078d8afd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:45.325833 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f88e8-ec03-4edc-a922-283078d8afd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:45.326101 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f88e8-ec03-4edc-a922-283078d8afd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:45.326741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f88e8-ec03-4edc-a922-283078d8afd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:45.328433 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f88e8-ec03-4edc-a922-283078d8afd9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:45.343783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 795/1597] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:45.347816 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f88e8-ec03-4edc-a922-283078d8afd9 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:45.349734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2a1f88e8-ec03-4edc-a922-283078d8afd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:45.364388 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f88e8-ec03-4edc-a922-283078d8afd9 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:45.371574 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b69cde-b5e9-4235-bb26-db0924349ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:45.372577 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b69cde-b5e9-4235-bb26-db0924349ccc admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:45.372964 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b69cde-b5e9-4235-bb26-db0924349ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:45.373306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b69cde-b5e9-4235-bb26-db0924349ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:45.374130 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b69cde-b5e9-4235-bb26-db0924349ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:45.376693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b69cde-b5e9-4235-bb26-db0924349ccc admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:45.390123 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 796/1598] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:45.394518 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b69cde-b5e9-4235-bb26-db0924349ccc admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:45.402714 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 803/1599] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:45.406309 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b69cde-b5e9-4235-bb26-db0924349ccc admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:45.410514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e61e78-5829-4930-889f-3c39bbf10f9e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:45.410930 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e61e78-5829-4930-889f-3c39bbf10f9e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:45.411281 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e61e78-5829-4930-889f-3c39bbf10f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:45.419189 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556437c0-0e04-45bf-8f3b-eea4c70febc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:45.419865 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556437c0-0e04-45bf-8f3b-eea4c70febc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:45.420147 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556437c0-0e04-45bf-8f3b-eea4c70febc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:45.420421 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556437c0-0e04-45bf-8f3b-eea4c70febc4 admin admin] PATH_INFO: `/v3/projects/fe626ac29e53476d8016ee98dea3a9b1/users/19fe1c8eb65d477b99e6c79dce57fc51/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:45.438444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-13e61e78-5829-4930-889f-3c39bbf10f9e None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:45.440623 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-13e61e78-5829-4930-889f-3c39bbf10f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:45.451817 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556437c0-0e04-45bf-8f3b-eea4c70febc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe626ac29e53476d8016ee98dea3a9b1, user_id=19fe1c8eb65d477b99e6c79dce57fc51, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:45.453930 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 804/1600] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:45.454974 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556437c0-0e04-45bf-8f3b-eea4c70febc4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:45.474941 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e61e78-5829-4930-889f-3c39bbf10f9e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:45.485438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 797/1601] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:45 2022] PUT /identity/v3/projects/fe626ac29e53476d8016ee98dea3a9b1/users/19fe1c8eb65d477b99e6c79dce57fc51/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:12:45.489810 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e61e78-5829-4930-889f-3c39bbf10f9e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:45.495959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:45.498384 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c87f4-2830-42b7-9ac6-9c3f53dccef6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:45.498834 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c87f4-2830-42b7-9ac6-9c3f53dccef6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:45.499582 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c87f4-2830-42b7-9ac6-9c3f53dccef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:45.551015 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-318c87f4-2830-42b7-9ac6-9c3f53dccef6 None None] MFA Rules not processed for user `19fe1c8eb65d477b99e6c79dce57fc51`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:45.552717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-318c87f4-2830-42b7-9ac6-9c3f53dccef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:45.609432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c31af-73da-4406-8dfe-e09a7b7c400f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:45.610555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c31af-73da-4406-8dfe-e09a7b7c400f admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:45.610858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c31af-73da-4406-8dfe-e09a7b7c400f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:45.611160 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c31af-73da-4406-8dfe-e09a7b7c400f admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:45.611818 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c31af-73da-4406-8dfe-e09a7b7c400f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:12:45.612272 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c31af-73da-4406-8dfe-e09a7b7c400f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:45.614417 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c31af-73da-4406-8dfe-e09a7b7c400f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:45.630608 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 805/1602] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:12:45.641726 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 798/1603] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:12:45.647273 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c31af-73da-4406-8dfe-e09a7b7c400f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:45.647441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c87f4-2830-42b7-9ac6-9c3f53dccef6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:45.663348 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c31af-73da-4406-8dfe-e09a7b7c400f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:45.665317 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c87f4-2830-42b7-9ac6-9c3f53dccef6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:45.670306 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22385f4-4545-4356-99e0-8f6c1305e8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:45.670997 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22385f4-4545-4356-99e0-8f6c1305e8a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:45.671281 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22385f4-4545-4356-99e0-8f6c1305e8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:45.671553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22385f4-4545-4356-99e0-8f6c1305e8a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:45.672320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22385f4-4545-4356-99e0-8f6c1305e8a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:45.674011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c4bf8-3d82-4e47-a939-0165c56ea3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:45.674887 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c4bf8-3d82-4e47-a939-0165c56ea3fb admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:45.675008 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22385f4-4545-4356-99e0-8f6c1305e8a3 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:45.675476 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c4bf8-3d82-4e47-a939-0165c56ea3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:45.676257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c4bf8-3d82-4e47-a939-0165c56ea3fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:45.677103 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c4bf8-3d82-4e47-a939-0165c56ea3fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:45.680863 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c4bf8-3d82-4e47-a939-0165c56ea3fb admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:45.728262 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:45.756369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 799/1604] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:45 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:45.759394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c4bf8-3d82-4e47-a939-0165c56ea3fb admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:45.766270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 806/1605] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:45 2022] POST /identity/v3/projects => generated 389 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:45.768398 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22385f4-4545-4356-99e0-8f6c1305e8a3 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:45.925389 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c4bf8-3d82-4e47-a939-0165c56ea3fb admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:45.935779 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22385f4-4545-4356-99e0-8f6c1305e8a3 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:45.947366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1ad6b-ab1a-43b4-86ba-3bd71a3daed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:45.948812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b6c20-fc3a-4a70-8e9d-a344dab3b5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:45.949850 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b6c20-fc3a-4a70-8e9d-a344dab3b5bd admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:45.951134 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1ad6b-ab1a-43b4-86ba-3bd71a3daed8 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:45.951431 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1ad6b-ab1a-43b4-86ba-3bd71a3daed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:45.951701 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1ad6b-ab1a-43b4-86ba-3bd71a3daed8 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:45.952256 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b6c20-fc3a-4a70-8e9d-a344dab3b5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:45.952821 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b6c20-fc3a-4a70-8e9d-a344dab3b5bd admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:45.954779 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b6c20-fc3a-4a70-8e9d-a344dab3b5bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:45.958594 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c1ad6b-ab1a-43b4-86ba-3bd71a3daed8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:45.961817 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c1ad6b-ab1a-43b4-86ba-3bd71a3daed8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:45.970116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b6c20-fc3a-4a70-8e9d-a344dab3b5bd admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:46.022840 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 800/1606] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:45 2022] POST /identity/v3/users => generated 293 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:46.028112 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1ad6b-ab1a-43b4-86ba-3bd71a3daed8 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:46.048665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 807/1607] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:45 2022] POST /identity/v3/users => generated 295 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:46.048794 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1ad6b-ab1a-43b4-86ba-3bd71a3daed8 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:46.055288 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b6c20-fc3a-4a70-8e9d-a344dab3b5bd admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:46.064246 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3569c-130f-4181-9466-1cead0b3efad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:46.066952 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3569c-130f-4181-9466-1cead0b3efad admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:46.067460 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3569c-130f-4181-9466-1cead0b3efad admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:46.069327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3569c-130f-4181-9466-1cead0b3efad admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:46.070169 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3569c-130f-4181-9466-1cead0b3efad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:46.074986 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3569c-130f-4181-9466-1cead0b3efad admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:46.076378 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b6c20-fc3a-4a70-8e9d-a344dab3b5bd admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:46.087379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab361b0-0bef-4c6e-8f71-48b436c8e65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:46.088009 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab361b0-0bef-4c6e-8f71-48b436c8e65b admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:46.088290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab361b0-0bef-4c6e-8f71-48b436c8e65b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:46.088563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab361b0-0bef-4c6e-8f71-48b436c8e65b admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:46.089629 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab361b0-0bef-4c6e-8f71-48b436c8e65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:46.090802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab361b0-0bef-4c6e-8f71-48b436c8e65b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:46.103430 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 801/1608] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:46.104353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3569c-130f-4181-9466-1cead0b3efad admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:46.109373 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 808/1609] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:12:46.110575 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab361b0-0bef-4c6e-8f71-48b436c8e65b admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:46.112870 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7fa3569c-130f-4181-9466-1cead0b3efad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:46.136749 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab361b0-0bef-4c6e-8f71-48b436c8e65b admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:46.151328 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d4ece-721d-40fc-9584-7ccf8e39808d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:46.152219 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d4ece-721d-40fc-9584-7ccf8e39808d admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:46.152501 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d4ece-721d-40fc-9584-7ccf8e39808d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:46.152774 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d4ece-721d-40fc-9584-7ccf8e39808d admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:46.153422 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d4ece-721d-40fc-9584-7ccf8e39808d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:46.155854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d4ece-721d-40fc-9584-7ccf8e39808d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:46.174201 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 809/1610] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:46.179017 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d4ece-721d-40fc-9584-7ccf8e39808d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:46.189493 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d4ece-721d-40fc-9584-7ccf8e39808d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:46.194783 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf92ac-157d-4296-a204-ddf3020ffcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:46.195487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf92ac-157d-4296-a204-ddf3020ffcaf admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:46.196508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf92ac-157d-4296-a204-ddf3020ffcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:46.196508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf92ac-157d-4296-a204-ddf3020ffcaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:46.196863 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf92ac-157d-4296-a204-ddf3020ffcaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:46.198628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf92ac-157d-4296-a204-ddf3020ffcaf admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:46.210919 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 810/1611] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:46.210919 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf92ac-157d-4296-a204-ddf3020ffcaf admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:46.228349 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf92ac-157d-4296-a204-ddf3020ffcaf admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:46.233333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b712f-466d-427d-aa2c-dccbba5f3a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:46.234301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b712f-466d-427d-aa2c-dccbba5f3a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:46.234687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b712f-466d-427d-aa2c-dccbba5f3a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:46.235111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b712f-466d-427d-aa2c-dccbba5f3a7b admin admin] PATH_INFO: `/v3/projects/a7bbf740783e495981ad50749641174c/users/dec10d9dd14d44739f7bd710d29da065/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:46.257853 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b712f-466d-427d-aa2c-dccbba5f3a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bbf740783e495981ad50749641174c, user_id=dec10d9dd14d44739f7bd710d29da065, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:46.262274 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b712f-466d-427d-aa2c-dccbba5f3a7b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:46.262434 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3569c-130f-4181-9466-1cead0b3efad admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:46.298914 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 811/1612] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:46 2022] PUT /identity/v3/projects/a7bbf740783e495981ad50749641174c/users/dec10d9dd14d44739f7bd710d29da065/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:46.298914 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b712f-466d-427d-aa2c-dccbba5f3a7b admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:46.299230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69449e70-f4eb-4342-850d-fcdd7c047bf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:46.299230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69449e70-f4eb-4342-850d-fcdd7c047bf0 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:46.299230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69449e70-f4eb-4342-850d-fcdd7c047bf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:46.299230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69449e70-f4eb-4342-850d-fcdd7c047bf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:46.307198 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-69449e70-f4eb-4342-850d-fcdd7c047bf0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:46.347019 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:46.415153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69449e70-f4eb-4342-850d-fcdd7c047bf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:46.416750 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69449e70-f4eb-4342-850d-fcdd7c047bf0 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:46.428554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 802/1613] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:46.431229 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728f64a-33d5-4353-8b04-2a938dc1179c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:46.431672 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728f64a-33d5-4353-8b04-2a938dc1179c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:46.432093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728f64a-33d5-4353-8b04-2a938dc1179c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:46.465569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b712f-466d-427d-aa2c-dccbba5f3a7b admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:46.467746 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e728f64a-33d5-4353-8b04-2a938dc1179c None None] MFA Rules not processed for user `dec10d9dd14d44739f7bd710d29da065`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:46.469095 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e728f64a-33d5-4353-8b04-2a938dc1179c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:46.473331 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397e8a1-353c-4549-b4bf-18d9a204bf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:46.474364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397e8a1-353c-4549-b4bf-18d9a204bf7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:46.474851 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397e8a1-353c-4549-b4bf-18d9a204bf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:46.475941 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397e8a1-353c-4549-b4bf-18d9a204bf7c admin admin] PATH_INFO: `/v3/projects/23e3b0cfae654d9f8e0e82cf631df127/users/d233c31ec6514247af71b22c9380bbd8/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:46.497746 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397e8a1-353c-4549-b4bf-18d9a204bf7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e3b0cfae654d9f8e0e82cf631df127, user_id=d233c31ec6514247af71b22c9380bbd8, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:46.499844 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:46.502174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397e8a1-353c-4549-b4bf-18d9a204bf7c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:46.533674 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 812/1614] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:46 2022] PUT /identity/v3/projects/23e3b0cfae654d9f8e0e82cf631df127/users/d233c31ec6514247af71b22c9380bbd8/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:46.545127 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c72e5-f11b-49d6-ae6e-63b3c00c5366 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:46.545403 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c72e5-f11b-49d6-ae6e-63b3c00c5366 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:46.546089 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c72e5-f11b-49d6-ae6e-63b3c00c5366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:46.566018 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 803/1615] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:46.571664 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728f64a-33d5-4353-8b04-2a938dc1179c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:46.587591 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728f64a-33d5-4353-8b04-2a938dc1179c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:46.595309 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-fc5c72e5-f11b-49d6-ae6e-63b3c00c5366 None None] MFA Rules not processed for user `d233c31ec6514247af71b22c9380bbd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:46.596830 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fc5c72e5-f11b-49d6-ae6e-63b3c00c5366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:46.713758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 813/1616] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:46.724176 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c72e5-f11b-49d6-ae6e-63b3c00c5366 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:46.741306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc383b-24db-45ed-a8f2-61fe68021394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:46.742138 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc383b-24db-45ed-a8f2-61fe68021394 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:46.742323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc383b-24db-45ed-a8f2-61fe68021394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:46.742773 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc383b-24db-45ed-a8f2-61fe68021394 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:46.743336 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc383b-24db-45ed-a8f2-61fe68021394 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:46.748870 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc383b-24db-45ed-a8f2-61fe68021394 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:46.767031 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c72e5-f11b-49d6-ae6e-63b3c00c5366 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:46.781664 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364846f2-ee28-4c0d-89c2-933cedba1cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:46.782354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364846f2-ee28-4c0d-89c2-933cedba1cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:46.782727 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364846f2-ee28-4c0d-89c2-933cedba1cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:46.783060 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364846f2-ee28-4c0d-89c2-933cedba1cc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:46.783779 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364846f2-ee28-4c0d-89c2-933cedba1cc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:46.786229 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364846f2-ee28-4c0d-89c2-933cedba1cc9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:46.859976 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 804/1617] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:46 2022] POST /identity/v3/projects => generated 385 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:12:46.869089 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc383b-24db-45ed-a8f2-61fe68021394 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:46.899720 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 814/1618] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:46 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:46.901228 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364846f2-ee28-4c0d-89c2-933cedba1cc9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:47.013681 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc383b-24db-45ed-a8f2-61fe68021394 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:47.110017 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364846f2-ee28-4c0d-89c2-933cedba1cc9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:47.133273 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:47.135236 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fd112-7f5c-4984-b1d4-e655cdf8641b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:47.135784 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fd112-7f5c-4984-b1d4-e655cdf8641b admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:47.136235 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fd112-7f5c-4984-b1d4-e655cdf8641b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:47.136749 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fd112-7f5c-4984-b1d4-e655cdf8641b admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:47.138143 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267fd112-7f5c-4984-b1d4-e655cdf8641b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:47.144436 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267fd112-7f5c-4984-b1d4-e655cdf8641b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:47.154936 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e49db-b667-49c2-9bfa-c28bff4e077a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:47.156080 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e49db-b667-49c2-9bfa-c28bff4e077a admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:47.156367 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e49db-b667-49c2-9bfa-c28bff4e077a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:47.156642 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e49db-b667-49c2-9bfa-c28bff4e077a admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:47.157340 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e49db-b667-49c2-9bfa-c28bff4e077a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:47.159808 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:47.161279 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e49db-b667-49c2-9bfa-c28bff4e077a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:47.185102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 815/1619] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:46 2022] POST /identity/v3/users => generated 300 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:47.190767 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fd112-7f5c-4984-b1d4-e655cdf8641b admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:47.212900 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fd112-7f5c-4984-b1d4-e655cdf8641b admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:47.215556 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 805/1620] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:46 2022] POST /identity/v3/users => generated 299 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:47.217250 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e49db-b667-49c2-9bfa-c28bff4e077a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:47.233363 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5db0e-00db-44fe-a53b-f4bee8553e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:47.234139 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5db0e-00db-44fe-a53b-f4bee8553e09 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:47.234458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5db0e-00db-44fe-a53b-f4bee8553e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:47.234729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5db0e-00db-44fe-a53b-f4bee8553e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:47.235400 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5db0e-00db-44fe-a53b-f4bee8553e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:47.235532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e49db-b667-49c2-9bfa-c28bff4e077a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:47.237010 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5db0e-00db-44fe-a53b-f4bee8553e09 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:47.243266 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e9951-2c95-4e6c-9e71-d9faedfebdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:47.244100 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e9951-2c95-4e6c-9e71-d9faedfebdb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:47.244388 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e9951-2c95-4e6c-9e71-d9faedfebdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:47.244727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e9951-2c95-4e6c-9e71-d9faedfebdb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:47.245396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e9951-2c95-4e6c-9e71-d9faedfebdb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:47.248508 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e9951-2c95-4e6c-9e71-d9faedfebdb7 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:47.265872 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 816/1621] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:47.271245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5db0e-00db-44fe-a53b-f4bee8553e09 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:47.284886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5db0e-00db-44fe-a53b-f4bee8553e09 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:47.286574 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 806/1622] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:47.291025 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a16556-808a-4e16-9450-5cae771ca8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:47.291858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a16556-808a-4e16-9450-5cae771ca8bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:47.292146 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a16556-808a-4e16-9450-5cae771ca8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:47.292482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a16556-808a-4e16-9450-5cae771ca8bd admin admin] PATH_INFO: `/v3/projects/fce024ccace94218b4f70020217f31f2/users/5ccc90d5ba204d69abe999c5da1f21f5/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:47.293101 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e9951-2c95-4e6c-9e71-d9faedfebdb7 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:47.309888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e9951-2c95-4e6c-9e71-d9faedfebdb7 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:47.311754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a16556-808a-4e16-9450-5cae771ca8bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fce024ccace94218b4f70020217f31f2, user_id=5ccc90d5ba204d69abe999c5da1f21f5, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:47.315826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd62a4c-68cf-4b78-a23d-1909c513009d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:47.315987 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a16556-808a-4e16-9450-5cae771ca8bd admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:47.316939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd62a4c-68cf-4b78-a23d-1909c513009d admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:47.317388 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd62a4c-68cf-4b78-a23d-1909c513009d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:47.317830 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd62a4c-68cf-4b78-a23d-1909c513009d admin admin] PATH_INFO: `/v3/projects/74deef577f8e47b19776fa862ae13339/users/6d2bc5750b00497aabe85a022c945eb5/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:47.347228 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 817/1623] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:47 2022] PUT /identity/v3/projects/fce024ccace94218b4f70020217f31f2/users/5ccc90d5ba204d69abe999c5da1f21f5/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:47.347515 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd62a4c-68cf-4b78-a23d-1909c513009d admin admin] RBAC: Authorizing `identity:create_grant(project_id=74deef577f8e47b19776fa862ae13339, user_id=6d2bc5750b00497aabe85a022c945eb5, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:47.347996 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd62a4c-68cf-4b78-a23d-1909c513009d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:47.349373 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a16556-808a-4e16-9450-5cae771ca8bd admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:47.377153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 807/1624] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:47 2022] PUT /identity/v3/projects/74deef577f8e47b19776fa862ae13339/users/6d2bc5750b00497aabe85a022c945eb5/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:47.383338 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd62a4c-68cf-4b78-a23d-1909c513009d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:47.455577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a16556-808a-4e16-9450-5cae771ca8bd admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:47.502303 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd62a4c-68cf-4b78-a23d-1909c513009d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:47.511668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92d0b7-52c5-401f-bbbd-329a80805273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:47.511668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92d0b7-52c5-401f-bbbd-329a80805273 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:47.512289 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92d0b7-52c5-401f-bbbd-329a80805273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:47.513103 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92d0b7-52c5-401f-bbbd-329a80805273 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:47.516658 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92d0b7-52c5-401f-bbbd-329a80805273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:47.521402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92d0b7-52c5-401f-bbbd-329a80805273 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:47.550054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 808/1625] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:47.551101 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e86dac-05ac-4d11-865c-157a07efbdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:47.552632 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92d0b7-52c5-401f-bbbd-329a80805273 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:47.554137 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e86dac-05ac-4d11-865c-157a07efbdf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:47.554408 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e86dac-05ac-4d11-865c-157a07efbdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:47.555041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e86dac-05ac-4d11-865c-157a07efbdf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:47.555732 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e86dac-05ac-4d11-865c-157a07efbdf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:47.557681 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e86dac-05ac-4d11-865c-157a07efbdf3 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:47.565605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92d0b7-52c5-401f-bbbd-329a80805273 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:47.570917 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 818/1626] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:47.571495 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdca861-5778-4a1d-bf31-1a525acd0f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:47.572182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdca861-5778-4a1d-bf31-1a525acd0f99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:47.572472 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdca861-5778-4a1d-bf31-1a525acd0f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:47.572907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdca861-5778-4a1d-bf31-1a525acd0f99 admin admin] PATH_INFO: `/v3/projects/74deef577f8e47b19776fa862ae13339/users/6d2bc5750b00497aabe85a022c945eb5/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:47.574040 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e86dac-05ac-4d11-865c-157a07efbdf3 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:47.575025 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdca861-5778-4a1d-bf31-1a525acd0f99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74deef577f8e47b19776fa862ae13339, user_id=6d2bc5750b00497aabe85a022c945eb5, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:47.577753 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdca861-5778-4a1d-bf31-1a525acd0f99 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:47.590919 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 809/1627] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:47 2022] PUT /identity/v3/projects/74deef577f8e47b19776fa862ae13339/users/6d2bc5750b00497aabe85a022c945eb5/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:47.591811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e86dac-05ac-4d11-865c-157a07efbdf3 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:47.601515 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30593b5d-e7a5-4972-9a72-7826c2f492e3 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:47.601829 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30593b5d-e7a5-4972-9a72-7826c2f492e3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:47.602116 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30593b5d-e7a5-4972-9a72-7826c2f492e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:47.638953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-30593b5d-e7a5-4972-9a72-7826c2f492e3 None None] MFA Rules not processed for user `6d2bc5750b00497aabe85a022c945eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:47.641315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-30593b5d-e7a5-4972-9a72-7826c2f492e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:47.721569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb8623-472b-4202-a289-bad3315369dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:47.723622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb8623-472b-4202-a289-bad3315369dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:47.724245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb8623-472b-4202-a289-bad3315369dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:47.727262 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb8623-472b-4202-a289-bad3315369dc admin admin] PATH_INFO: `/v3/projects/fce024ccace94218b4f70020217f31f2/users/5ccc90d5ba204d69abe999c5da1f21f5/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:47.733106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb8623-472b-4202-a289-bad3315369dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fce024ccace94218b4f70020217f31f2, user_id=5ccc90d5ba204d69abe999c5da1f21f5, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:47.737174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb8623-472b-4202-a289-bad3315369dc admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:47.745886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:47.785412 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 819/1628] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:47 2022] PUT /identity/v3/projects/fce024ccace94218b4f70020217f31f2/users/5ccc90d5ba204d69abe999c5da1f21f5/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:47.791039 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142934a1-cb86-4146-9357-6a7c33a643bd None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:47.791353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142934a1-cb86-4146-9357-6a7c33a643bd None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:47.791569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142934a1-cb86-4146-9357-6a7c33a643bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:47.800733 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 810/1629] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:47.823495 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30593b5d-e7a5-4972-9a72-7826c2f492e3 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:47.825358 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-30593b5d-e7a5-4972-9a72-7826c2f492e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:47.834842 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-142934a1-cb86-4146-9357-6a7c33a643bd None None] MFA Rules not processed for user `5ccc90d5ba204d69abe999c5da1f21f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:47.838191 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-142934a1-cb86-4146-9357-6a7c33a643bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:47.846623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:47.897099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30593b5d-e7a5-4972-9a72-7826c2f492e3 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:47.932992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 820/1630] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:47 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:47.941305 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142934a1-cb86-4146-9357-6a7c33a643bd None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:47.971904 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956efd9-6991-4a07-9f1b-e2ae60e01065 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:47.972530 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956efd9-6991-4a07-9f1b-e2ae60e01065 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:47.972809 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956efd9-6991-4a07-9f1b-e2ae60e01065 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:47.973085 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956efd9-6991-4a07-9f1b-e2ae60e01065 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:48.026656 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142934a1-cb86-4146-9357-6a7c33a643bd None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:48.033405 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee712b2f-4a16-4a54-8487-505739e99558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:48.034169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee712b2f-4a16-4a54-8487-505739e99558 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:48.034558 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee712b2f-4a16-4a54-8487-505739e99558 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:48.034995 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee712b2f-4a16-4a54-8487-505739e99558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:48.049075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956efd9-6991-4a07-9f1b-e2ae60e01065 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:48.050678 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956efd9-6991-4a07-9f1b-e2ae60e01065 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:48.053746 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee712b2f-4a16-4a54-8487-505739e99558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:48.055397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee712b2f-4a16-4a54-8487-505739e99558 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:48.063127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 811/1631] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:48.078305 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 821/1632] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:12:48.684035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956efd9-6991-4a07-9f1b-e2ae60e01065 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:48.769865 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956efd9-6991-4a07-9f1b-e2ae60e01065 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:48.776227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9721a7-f9e9-4199-a290-16796996e55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:48.777323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9721a7-f9e9-4199-a290-16796996e55d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:48.777673 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9721a7-f9e9-4199-a290-16796996e55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:48.778786 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9721a7-f9e9-4199-a290-16796996e55d admin admin] PATH_INFO: `/v3/users/d233c31ec6514247af71b22c9380bbd8` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:48.784497 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9721a7-f9e9-4199-a290-16796996e55d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d233c31ec6514247af71b22c9380bbd8)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:48.788274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9721a7-f9e9-4199-a290-16796996e55d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:48.840613 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:48.862693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5d9721a7-f9e9-4199-a290-16796996e55d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd233c31ec6514247af71b22c9380bbd8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:48.871584 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5d9721a7-f9e9-4199-a290-16796996e55d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd233c31ec6514247af71b22c9380bbd8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:48.872702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5d9721a7-f9e9-4199-a290-16796996e55d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd233c31ec6514247af71b22c9380bbd8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:48.881431 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5d9721a7-f9e9-4199-a290-16796996e55d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd233c31ec6514247af71b22c9380bbd8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:48.894639 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5d9721a7-f9e9-4199-a290-16796996e55d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd233c31ec6514247af71b22c9380bbd8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:48.902231 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 812/1633] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:48 2022] DELETE /identity/v3/users/d233c31ec6514247af71b22c9380bbd8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:48.906902 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d028c9d-2737-448b-8064-41ef7b3acade None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:48.907425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d028c9d-2737-448b-8064-41ef7b3acade None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:48.907564 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d028c9d-2737-448b-8064-41ef7b3acade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:48.927759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3d028c9d-2737-448b-8064-41ef7b3acade None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:48.929112 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3d028c9d-2737-448b-8064-41ef7b3acade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:48.994545 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 822/1634] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:49.004258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9721a7-f9e9-4199-a290-16796996e55d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:49.005665 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5d9721a7-f9e9-4199-a290-16796996e55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:49.069567 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9721a7-f9e9-4199-a290-16796996e55d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:49.073194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115a01a-e203-494a-8477-e2700dd83850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:49.073805 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115a01a-e203-494a-8477-e2700dd83850 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:49.074597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115a01a-e203-494a-8477-e2700dd83850 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:49.074894 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7115a01a-e203-494a-8477-e2700dd83850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:49.084291 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115a01a-e203-494a-8477-e2700dd83850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:49.086249 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7115a01a-e203-494a-8477-e2700dd83850 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:49.096412 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 813/1635] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:49.473984 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d028c9d-2737-448b-8064-41ef7b3acade None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:49.477117 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3d028c9d-2737-448b-8064-41ef7b3acade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:49.513207 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d028c9d-2737-448b-8064-41ef7b3acade None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:49.525311 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5f539-d820-4405-a974-e2b126075cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:49.525887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5f539-d820-4405-a974-e2b126075cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:49.526169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5f539-d820-4405-a974-e2b126075cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:49.526491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5f539-d820-4405-a974-e2b126075cbd admin admin] PATH_INFO: `/v3/projects/23e3b0cfae654d9f8e0e82cf631df127` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:49.533133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5f539-d820-4405-a974-e2b126075cbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=23e3b0cfae654d9f8e0e82cf631df127)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:49.535190 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5f539-d820-4405-a974-e2b126075cbd admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:49.554951 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:49.606364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6af5f539-d820-4405-a974-e2b126075cbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23e3b0cfae654d9f8e0e82cf631df127', 'request_id': 'req-6af5f539-d820-4405-a974-e2b126075cbd'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:49.613806 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6af5f539-d820-4405-a974-e2b126075cbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23e3b0cfae654d9f8e0e82cf631df127', 'request_id': 'req-6af5f539-d820-4405-a974-e2b126075cbd'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:49.623947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6af5f539-d820-4405-a974-e2b126075cbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23e3b0cfae654d9f8e0e82cf631df127', 'request_id': 'req-6af5f539-d820-4405-a974-e2b126075cbd'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:49.641571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6af5f539-d820-4405-a974-e2b126075cbd admin admin] The token cache is being invalidate because project 23e3b0cfae654d9f8e0e82cf631df127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:49.642078 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6af5f539-d820-4405-a974-e2b126075cbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:49.642843 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6af5f539-d820-4405-a974-e2b126075cbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:49.645634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 823/1636] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:49 2022] DELETE /identity/v3/projects/23e3b0cfae654d9f8e0e82cf631df127 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:49.650189 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115a01a-e203-494a-8477-e2700dd83850 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:49.664898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7115a01a-e203-494a-8477-e2700dd83850 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:49.752687 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f3016-4518-412b-b0d5-a5b77be3153c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:49.753114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f3016-4518-412b-b0d5-a5b77be3153c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:49.753395 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f3016-4518-412b-b0d5-a5b77be3153c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:49.753875 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f3016-4518-412b-b0d5-a5b77be3153c admin admin] PATH_INFO: `/v3/users/5ccc90d5ba204d69abe999c5da1f21f5` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:49.754855 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f3016-4518-412b-b0d5-a5b77be3153c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ccc90d5ba204d69abe999c5da1f21f5)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:49.757494 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f3016-4518-412b-b0d5-a5b77be3153c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:49.833279 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0a5f3016-4518-412b-b0d5-a5b77be3153c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ccc90d5ba204d69abe999c5da1f21f5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:49.836597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:49.852791 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0a5f3016-4518-412b-b0d5-a5b77be3153c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ccc90d5ba204d69abe999c5da1f21f5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:49.854341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0a5f3016-4518-412b-b0d5-a5b77be3153c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ccc90d5ba204d69abe999c5da1f21f5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:49.864762 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0a5f3016-4518-412b-b0d5-a5b77be3153c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ccc90d5ba204d69abe999c5da1f21f5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:49.881175 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0a5f3016-4518-412b-b0d5-a5b77be3153c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ccc90d5ba204d69abe999c5da1f21f5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:49.889777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 814/1637] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:49 2022] DELETE /identity/v3/users/5ccc90d5ba204d69abe999c5da1f21f5 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:50.426919 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5f539-d820-4405-a974-e2b126075cbd admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:50.429928 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6af5f539-d820-4405-a974-e2b126075cbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:50.513122 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5f539-d820-4405-a974-e2b126075cbd admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:50.588760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b7eff0-5287-4859-a49b-6ab4429fa3e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:50.589405 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7eff0-5287-4859-a49b-6ab4429fa3e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:50.589687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7eff0-5287-4859-a49b-6ab4429fa3e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:50.589955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b7eff0-5287-4859-a49b-6ab4429fa3e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:50.592951 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d6b7eff0-5287-4859-a49b-6ab4429fa3e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:50.680862 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7eff0-5287-4859-a49b-6ab4429fa3e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:50.682836 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b7eff0-5287-4859-a49b-6ab4429fa3e2 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:50.710412 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 824/1638] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:51.205045 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f3016-4518-412b-b0d5-a5b77be3153c admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:51.208945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0a5f3016-4518-412b-b0d5-a5b77be3153c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:51.382052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f3016-4518-412b-b0d5-a5b77be3153c admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:51.395472 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b05ee-7922-4649-b556-c1234bd0105d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:51.396146 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b05ee-7922-4649-b556-c1234bd0105d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:51.396412 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b05ee-7922-4649-b556-c1234bd0105d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:51.396686 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b05ee-7922-4649-b556-c1234bd0105d admin admin] PATH_INFO: `/v3/projects/fce024ccace94218b4f70020217f31f2` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:51.399457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b05ee-7922-4649-b556-c1234bd0105d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fce024ccace94218b4f70020217f31f2)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:51.401394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b05ee-7922-4649-b556-c1234bd0105d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:51.529658 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-071b05ee-7922-4649-b556-c1234bd0105d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fce024ccace94218b4f70020217f31f2', 'request_id': 'req-071b05ee-7922-4649-b556-c1234bd0105d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:51.556277 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-071b05ee-7922-4649-b556-c1234bd0105d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fce024ccace94218b4f70020217f31f2', 'request_id': 'req-071b05ee-7922-4649-b556-c1234bd0105d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:51.576200 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-071b05ee-7922-4649-b556-c1234bd0105d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fce024ccace94218b4f70020217f31f2', 'request_id': 'req-071b05ee-7922-4649-b556-c1234bd0105d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:51.596323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-071b05ee-7922-4649-b556-c1234bd0105d admin admin] The token cache is being invalidate because project fce024ccace94218b4f70020217f31f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:51.596632 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-071b05ee-7922-4649-b556-c1234bd0105d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:51.598075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-071b05ee-7922-4649-b556-c1234bd0105d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:51.602961 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 815/1639] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:51 2022] DELETE /identity/v3/projects/fce024ccace94218b4f70020217f31f2 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:51.607807 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be62c10-0a4f-414b-8a5a-9567852b3e2f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:51.608729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be62c10-0a4f-414b-8a5a-9567852b3e2f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:51.609011 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be62c10-0a4f-414b-8a5a-9567852b3e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:51.661667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3be62c10-0a4f-414b-8a5a-9567852b3e2f None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:51.665765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3be62c10-0a4f-414b-8a5a-9567852b3e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:51.690707 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b05ee-7922-4649-b556-c1234bd0105d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:51.692475 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-071b05ee-7922-4649-b556-c1234bd0105d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:51.694364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:51.799374 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 825/1640] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:51.811266 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf1191-aa19-4f38-85a0-3bfe3a1a0677 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:51.811566 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf1191-aa19-4f38-85a0-3bfe3a1a0677 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:51.811847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf1191-aa19-4f38-85a0-3bfe3a1a0677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:51.831209 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b05ee-7922-4649-b556-c1234bd0105d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:51.836096 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-64bf1191-aa19-4f38-85a0-3bfe3a1a0677 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:51.836903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c78220-fff6-474f-9645-60ac8f2461ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:51.837433 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-64bf1191-aa19-4f38-85a0-3bfe3a1a0677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:51.838961 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c78220-fff6-474f-9645-60ac8f2461ed service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:51.839603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c78220-fff6-474f-9645-60ac8f2461ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:51.840102 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c78220-fff6-474f-9645-60ac8f2461ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:51.852320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b2c78220-fff6-474f-9645-60ac8f2461ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:51.855747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 826/1641] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:51.856865 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf1191-aa19-4f38-85a0-3bfe3a1a0677 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:51.877844 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf1191-aa19-4f38-85a0-3bfe3a1a0677 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:51.886727 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7162e6-fb28-4d9d-98b9-855969364320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:51.887566 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7162e6-fb28-4d9d-98b9-855969364320 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:51.887966 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7162e6-fb28-4d9d-98b9-855969364320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:51.888343 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7162e6-fb28-4d9d-98b9-855969364320 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:51.889328 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7162e6-fb28-4d9d-98b9-855969364320 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:12:51.889742 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7162e6-fb28-4d9d-98b9-855969364320 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:51.889885 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:51.894365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7162e6-fb28-4d9d-98b9-855969364320 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:51.928872 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 827/1642] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:12:51.930154 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7162e6-fb28-4d9d-98b9-855969364320 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:51.934024 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8b7162e6-fb28-4d9d-98b9-855969364320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:51.969788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7162e6-fb28-4d9d-98b9-855969364320 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:51.980254 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22585b21-5191-4934-a941-56c4331c95a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:51.980926 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22585b21-5191-4934-a941-56c4331c95a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:51.981399 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22585b21-5191-4934-a941-56c4331c95a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:51.981705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22585b21-5191-4934-a941-56c4331c95a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:51.985618 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-22585b21-5191-4934-a941-56c4331c95a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:52.020569 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c78220-fff6-474f-9645-60ac8f2461ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:52.022689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c78220-fff6-474f-9645-60ac8f2461ed service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:52.045700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 816/1643] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:12:52.046657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c78220-fff6-474f-9645-60ac8f2461ed service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:52.068260 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c78220-fff6-474f-9645-60ac8f2461ed service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:52.082375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b7b40-2dfb-4666-9504-4b6ecdb04d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:52.083031 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b7b40-2dfb-4666-9504-4b6ecdb04d4d admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:52.083399 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b7b40-2dfb-4666-9504-4b6ecdb04d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:52.083575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b7b40-2dfb-4666-9504-4b6ecdb04d4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:52.084215 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b7b40-2dfb-4666-9504-4b6ecdb04d4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:52.086781 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b7b40-2dfb-4666-9504-4b6ecdb04d4d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:52.128220 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22585b21-5191-4934-a941-56c4331c95a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:52.132668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22585b21-5191-4934-a941-56c4331c95a3 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:52.143426 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 817/1644] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:52 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:52.148980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b7b40-2dfb-4666-9504-4b6ecdb04d4d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:52.150682 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 828/1645] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:52.336953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b7b40-2dfb-4666-9504-4b6ecdb04d4d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:52.354734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80f469-1905-478b-9103-a20fd07cfad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:52.357447 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80f469-1905-478b-9103-a20fd07cfad9 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:52.359416 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80f469-1905-478b-9103-a20fd07cfad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:52.359923 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80f469-1905-478b-9103-a20fd07cfad9 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:52.361969 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80f469-1905-478b-9103-a20fd07cfad9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:52.370200 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80f469-1905-478b-9103-a20fd07cfad9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:52.439331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 818/1646] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:52 2022] POST /identity/v3/users => generated 293 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:12:52.444601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22585b21-5191-4934-a941-56c4331c95a3 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:52.462187 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22585b21-5191-4934-a941-56c4331c95a3 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:52.473160 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747b6e6-d750-4a1b-9b95-d19045384a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:52.473862 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747b6e6-d750-4a1b-9b95-d19045384a4d admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:52.474084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747b6e6-d750-4a1b-9b95-d19045384a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:52.474316 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747b6e6-d750-4a1b-9b95-d19045384a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:52.478465 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747b6e6-d750-4a1b-9b95-d19045384a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:52.480346 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747b6e6-d750-4a1b-9b95-d19045384a4d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:52.500875 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 829/1647] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:52.504849 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80f469-1905-478b-9103-a20fd07cfad9 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:52.519812 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80f469-1905-478b-9103-a20fd07cfad9 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:52.530653 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d9417-ad02-43bc-866c-af147bd86b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:52.531431 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d9417-ad02-43bc-866c-af147bd86b50 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:52.531619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d9417-ad02-43bc-866c-af147bd86b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:52.531905 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d9417-ad02-43bc-866c-af147bd86b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:52.532544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d9417-ad02-43bc-866c-af147bd86b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:52.534281 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d9417-ad02-43bc-866c-af147bd86b50 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:52.553715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747b6e6-d750-4a1b-9b95-d19045384a4d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:52.559358 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 819/1648] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:52.565867 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747b6e6-d750-4a1b-9b95-d19045384a4d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:52.575172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039730bc-c995-41fa-a4c3-4c3f435071e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:52.575856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039730bc-c995-41fa-a4c3-4c3f435071e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:52.576390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039730bc-c995-41fa-a4c3-4c3f435071e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:52.576670 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039730bc-c995-41fa-a4c3-4c3f435071e9 admin admin] PATH_INFO: `/v3/projects/246520cd6cae4ce099d62e7fc85bf15d/users/675821fcad094531a09e620ac4594e7e/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:52.597531 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039730bc-c995-41fa-a4c3-4c3f435071e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=246520cd6cae4ce099d62e7fc85bf15d, user_id=675821fcad094531a09e620ac4594e7e, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:52.600957 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039730bc-c995-41fa-a4c3-4c3f435071e9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:52.620341 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 830/1649] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:52 2022] PUT /identity/v3/projects/246520cd6cae4ce099d62e7fc85bf15d/users/675821fcad094531a09e620ac4594e7e/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:52.630944 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db882f-76a7-495a-81d4-eecf034516a4 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:52.631852 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db882f-76a7-495a-81d4-eecf034516a4 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:52.632440 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db882f-76a7-495a-81d4-eecf034516a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:52.668296 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-82db882f-76a7-495a-81d4-eecf034516a4 None None] MFA Rules not processed for user `675821fcad094531a09e620ac4594e7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:52.673063 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-82db882f-76a7-495a-81d4-eecf034516a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:52.759909 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:52.779475 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 820/1650] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:12:52.788067 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039730bc-c995-41fa-a4c3-4c3f435071e9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:52.862183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039730bc-c995-41fa-a4c3-4c3f435071e9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:52.868870 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a77c2-e4b9-4fa8-8f02-c8da44b922cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:52.870597 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a77c2-e4b9-4fa8-8f02-c8da44b922cf service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:52.871191 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a77c2-e4b9-4fa8-8f02-c8da44b922cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:52.871620 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a77c2-e4b9-4fa8-8f02-c8da44b922cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:52.886761 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a77c2-e4b9-4fa8-8f02-c8da44b922cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:52.889053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a77c2-e4b9-4fa8-8f02-c8da44b922cf service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:52.911963 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 831/1651] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:54.636478 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db882f-76a7-495a-81d4-eecf034516a4 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:54.637895 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-82db882f-76a7-495a-81d4-eecf034516a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:54.702194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db882f-76a7-495a-81d4-eecf034516a4 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:54.706656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b527976-4b07-4a3e-ae51-5e4491e80c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:54.707313 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b527976-4b07-4a3e-ae51-5e4491e80c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:54.707586 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b527976-4b07-4a3e-ae51-5e4491e80c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:54.707854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b527976-4b07-4a3e-ae51-5e4491e80c8f admin admin] PATH_INFO: `/v3/users/98c6e2c0e175444db90947942aefe6a6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:54.708963 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b527976-4b07-4a3e-ae51-5e4491e80c8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=98c6e2c0e175444db90947942aefe6a6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:54.710923 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b527976-4b07-4a3e-ae51-5e4491e80c8f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:54.760060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7b527976-4b07-4a3e-ae51-5e4491e80c8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98c6e2c0e175444db90947942aefe6a6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:54.768439 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7b527976-4b07-4a3e-ae51-5e4491e80c8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98c6e2c0e175444db90947942aefe6a6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:54.769124 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7b527976-4b07-4a3e-ae51-5e4491e80c8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98c6e2c0e175444db90947942aefe6a6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:54.776788 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7b527976-4b07-4a3e-ae51-5e4491e80c8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98c6e2c0e175444db90947942aefe6a6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:54.790295 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7b527976-4b07-4a3e-ae51-5e4491e80c8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98c6e2c0e175444db90947942aefe6a6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:54.796284 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 821/1652] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:54 2022] DELETE /identity/v3/users/98c6e2c0e175444db90947942aefe6a6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:54.800830 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808168f7-5e65-4ab4-930c-bfa30c1d11e3 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:54.801319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808168f7-5e65-4ab4-930c-bfa30c1d11e3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:54.801396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808168f7-5e65-4ab4-930c-bfa30c1d11e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:54.809041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:54.822163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-808168f7-5e65-4ab4-930c-bfa30c1d11e3 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:54.823483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-808168f7-5e65-4ab4-930c-bfa30c1d11e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:54.889221 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 832/1653] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:12:54.899605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b527976-4b07-4a3e-ae51-5e4491e80c8f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:54.901045 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7b527976-4b07-4a3e-ae51-5e4491e80c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:54.961377 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b527976-4b07-4a3e-ae51-5e4491e80c8f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:54.965238 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebc4a7-de0d-41e2-ad8c-453b9c32eb3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:54.965850 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebc4a7-de0d-41e2-ad8c-453b9c32eb3d service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:54.966181 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebc4a7-de0d-41e2-ad8c-453b9c32eb3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:54.966560 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebc4a7-de0d-41e2-ad8c-453b9c32eb3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:54.976271 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebc4a7-de0d-41e2-ad8c-453b9c32eb3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:54.977751 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebc4a7-de0d-41e2-ad8c-453b9c32eb3d service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:54.987131 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 822/1654] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:55.271387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808168f7-5e65-4ab4-930c-bfa30c1d11e3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:55.273096 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-808168f7-5e65-4ab4-930c-bfa30c1d11e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:55.288411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808168f7-5e65-4ab4-930c-bfa30c1d11e3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:55.293182 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5dcb9-ef6e-4579-97fc-2bb719d28713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:55.293830 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5dcb9-ef6e-4579-97fc-2bb719d28713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:55.294109 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5dcb9-ef6e-4579-97fc-2bb719d28713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:55.294380 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5dcb9-ef6e-4579-97fc-2bb719d28713 admin admin] PATH_INFO: `/v3/projects/ee1a20f3616745d997c9358ccec1f327` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:55.296059 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5dcb9-ef6e-4579-97fc-2bb719d28713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee1a20f3616745d997c9358ccec1f327)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:55.297957 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5dcb9-ef6e-4579-97fc-2bb719d28713 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:55.348466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-75f5dcb9-ef6e-4579-97fc-2bb719d28713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee1a20f3616745d997c9358ccec1f327', 'request_id': 'req-75f5dcb9-ef6e-4579-97fc-2bb719d28713'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:55.354558 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-75f5dcb9-ef6e-4579-97fc-2bb719d28713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee1a20f3616745d997c9358ccec1f327', 'request_id': 'req-75f5dcb9-ef6e-4579-97fc-2bb719d28713'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:55.360903 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-75f5dcb9-ef6e-4579-97fc-2bb719d28713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee1a20f3616745d997c9358ccec1f327', 'request_id': 'req-75f5dcb9-ef6e-4579-97fc-2bb719d28713'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:55.374047 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-75f5dcb9-ef6e-4579-97fc-2bb719d28713 admin admin] The token cache is being invalidate because project ee1a20f3616745d997c9358ccec1f327 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:55.374339 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-75f5dcb9-ef6e-4579-97fc-2bb719d28713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:55.375007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-75f5dcb9-ef6e-4579-97fc-2bb719d28713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:55.376526 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 833/1655] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:55 2022] DELETE /identity/v3/projects/ee1a20f3616745d997c9358ccec1f327 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:55.380580 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebc4a7-de0d-41e2-ad8c-453b9c32eb3d service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:55.390352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebc4a7-de0d-41e2-ad8c-453b9c32eb3d service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:55.402354 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:55.448424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb2516-f55e-4059-9781-241f01d3494c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:55.449048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb2516-f55e-4059-9781-241f01d3494c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:55.449324 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb2516-f55e-4059-9781-241f01d3494c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:55.449591 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb2516-f55e-4059-9781-241f01d3494c admin admin] PATH_INFO: `/v3/users/4d69dc883be94945a24ae90fca730595` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:55.450689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb2516-f55e-4059-9781-241f01d3494c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d69dc883be94945a24ae90fca730595)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:55.452629 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb2516-f55e-4059-9781-241f01d3494c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:55.498712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-aadb2516-f55e-4059-9781-241f01d3494c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d69dc883be94945a24ae90fca730595'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:55.506386 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-aadb2516-f55e-4059-9781-241f01d3494c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d69dc883be94945a24ae90fca730595'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:55.507086 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-aadb2516-f55e-4059-9781-241f01d3494c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d69dc883be94945a24ae90fca730595'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:55.514111 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-aadb2516-f55e-4059-9781-241f01d3494c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d69dc883be94945a24ae90fca730595'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:55.526098 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-aadb2516-f55e-4059-9781-241f01d3494c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d69dc883be94945a24ae90fca730595'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:55.532247 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 823/1656] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:55 2022] DELETE /identity/v3/users/4d69dc883be94945a24ae90fca730595 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:55.818003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5dcb9-ef6e-4579-97fc-2bb719d28713 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:55.819732 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-75f5dcb9-ef6e-4579-97fc-2bb719d28713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:55.840001 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5dcb9-ef6e-4579-97fc-2bb719d28713 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:55.898694 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8e20b-790e-416c-8470-c459cc9c10ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:55.899365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e20b-790e-416c-8470-c459cc9c10ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:55.899651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e20b-790e-416c-8470-c459cc9c10ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:55.899926 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e20b-790e-416c-8470-c459cc9c10ac admin admin] PATH_INFO: `/v3/projects/cc017af0f3ad409b87a9555dcddf735e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:55.901040 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8e20b-790e-416c-8470-c459cc9c10ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc017af0f3ad409b87a9555dcddf735e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:55.903243 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8e20b-790e-416c-8470-c459cc9c10ac admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:55.934037 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:55.950823 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3fa8e20b-790e-416c-8470-c459cc9c10ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc017af0f3ad409b87a9555dcddf735e', 'request_id': 'req-3fa8e20b-790e-416c-8470-c459cc9c10ac'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:55.957177 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3fa8e20b-790e-416c-8470-c459cc9c10ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc017af0f3ad409b87a9555dcddf735e', 'request_id': 'req-3fa8e20b-790e-416c-8470-c459cc9c10ac'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:55.964346 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3fa8e20b-790e-416c-8470-c459cc9c10ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc017af0f3ad409b87a9555dcddf735e', 'request_id': 'req-3fa8e20b-790e-416c-8470-c459cc9c10ac'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:55.978715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3fa8e20b-790e-416c-8470-c459cc9c10ac admin admin] The token cache is being invalidate because project cc017af0f3ad409b87a9555dcddf735e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:55.979042 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3fa8e20b-790e-416c-8470-c459cc9c10ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:55.979671 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-3fa8e20b-790e-416c-8470-c459cc9c10ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:55.981184 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 834/1657] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:55 2022] DELETE /identity/v3/projects/cc017af0f3ad409b87a9555dcddf735e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:55.985186 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb2516-f55e-4059-9781-241f01d3494c admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:55.986592 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-aadb2516-f55e-4059-9781-241f01d3494c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:55.999731 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb2516-f55e-4059-9781-241f01d3494c admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:56.054942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916fb0b-ac2b-4cb2-9cef-2541179b87e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:56.055566 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916fb0b-ac2b-4cb2-9cef-2541179b87e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:56.055842 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916fb0b-ac2b-4cb2-9cef-2541179b87e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:56.056164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916fb0b-ac2b-4cb2-9cef-2541179b87e8 admin admin] PATH_INFO: `/v3/users/13377b7ace454154bc8bd06eb75b6198` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:56.057259 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916fb0b-ac2b-4cb2-9cef-2541179b87e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13377b7ace454154bc8bd06eb75b6198)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:56.059185 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916fb0b-ac2b-4cb2-9cef-2541179b87e8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:56.094687 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:56.105247 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5916fb0b-ac2b-4cb2-9cef-2541179b87e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13377b7ace454154bc8bd06eb75b6198'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:56.112817 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5916fb0b-ac2b-4cb2-9cef-2541179b87e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13377b7ace454154bc8bd06eb75b6198'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:56.113494 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5916fb0b-ac2b-4cb2-9cef-2541179b87e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13377b7ace454154bc8bd06eb75b6198'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:56.121121 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5916fb0b-ac2b-4cb2-9cef-2541179b87e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13377b7ace454154bc8bd06eb75b6198'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:56.133044 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5916fb0b-ac2b-4cb2-9cef-2541179b87e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13377b7ace454154bc8bd06eb75b6198'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:56.138897 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 824/1658] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:55 2022] DELETE /identity/v3/users/13377b7ace454154bc8bd06eb75b6198 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:56.411217 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8e20b-790e-416c-8470-c459cc9c10ac admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:56.413318 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3fa8e20b-790e-416c-8470-c459cc9c10ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:56.427280 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8e20b-790e-416c-8470-c459cc9c10ac admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:56.483533 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d4a45-d3b5-47d8-8a4c-5e60d475bee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:56.484167 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d4a45-d3b5-47d8-8a4c-5e60d475bee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:56.484474 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d4a45-d3b5-47d8-8a4c-5e60d475bee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:56.484743 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d4a45-d3b5-47d8-8a4c-5e60d475bee2 admin admin] PATH_INFO: `/v3/projects/abddd56fbc0244ba954215e74af097ce` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:56.485847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d4a45-d3b5-47d8-8a4c-5e60d475bee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abddd56fbc0244ba954215e74af097ce)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:56.487768 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d4a45-d3b5-47d8-8a4c-5e60d475bee2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:56.534356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-899d4a45-d3b5-47d8-8a4c-5e60d475bee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abddd56fbc0244ba954215e74af097ce', 'request_id': 'req-899d4a45-d3b5-47d8-8a4c-5e60d475bee2'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:56.540590 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-899d4a45-d3b5-47d8-8a4c-5e60d475bee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abddd56fbc0244ba954215e74af097ce', 'request_id': 'req-899d4a45-d3b5-47d8-8a4c-5e60d475bee2'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:56.546729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-899d4a45-d3b5-47d8-8a4c-5e60d475bee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abddd56fbc0244ba954215e74af097ce', 'request_id': 'req-899d4a45-d3b5-47d8-8a4c-5e60d475bee2'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:56.558697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-899d4a45-d3b5-47d8-8a4c-5e60d475bee2 admin admin] The token cache is being invalidate because project abddd56fbc0244ba954215e74af097ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:56.559017 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-899d4a45-d3b5-47d8-8a4c-5e60d475bee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:56.559637 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-899d4a45-d3b5-47d8-8a4c-5e60d475bee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:56.561084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 835/1659] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:56 2022] DELETE /identity/v3/projects/abddd56fbc0244ba954215e74af097ce => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:56.567033 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422cede-4ce3-4704-a63b-3e03ed7b7e5b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:56.567385 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422cede-4ce3-4704-a63b-3e03ed7b7e5b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:56.567725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422cede-4ce3-4704-a63b-3e03ed7b7e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:56.587835 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b422cede-4ce3-4704-a63b-3e03ed7b7e5b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:56.588962 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b422cede-4ce3-4704-a63b-3e03ed7b7e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:56.648798 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 825/1660] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:56.657422 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa84a42-63f6-4b92-9093-879a2a396e6d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:56.657796 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa84a42-63f6-4b92-9093-879a2a396e6d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:56.658073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa84a42-63f6-4b92-9093-879a2a396e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:56.677632 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-1fa84a42-63f6-4b92-9093-879a2a396e6d None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:56.679426 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1fa84a42-63f6-4b92-9093-879a2a396e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:56.687284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 836/1661] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:56.691512 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422cede-4ce3-4704-a63b-3e03ed7b7e5b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:56.701297 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422cede-4ce3-4704-a63b-3e03ed7b7e5b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:56.705915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a1c60-5ebe-4024-b02c-2248b3269dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:56.706511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a1c60-5ebe-4024-b02c-2248b3269dea admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:56.706790 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a1c60-5ebe-4024-b02c-2248b3269dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:56.707090 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a1c60-5ebe-4024-b02c-2248b3269dea admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:56.707742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a1c60-5ebe-4024-b02c-2248b3269dea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:12:56.708041 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a1c60-5ebe-4024-b02c-2248b3269dea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:56.709634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a1c60-5ebe-4024-b02c-2248b3269dea admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:56.721648 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 826/1662] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:12:56.726487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa84a42-63f6-4b92-9093-879a2a396e6d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:56.735508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa84a42-63f6-4b92-9093-879a2a396e6d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:56.740060 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90634610-4073-434d-a82c-74bf1e9bd76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:56.740712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90634610-4073-434d-a82c-74bf1e9bd76f admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:56.740996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90634610-4073-434d-a82c-74bf1e9bd76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:56.741264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90634610-4073-434d-a82c-74bf1e9bd76f admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:56.741901 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90634610-4073-434d-a82c-74bf1e9bd76f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:56.743855 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90634610-4073-434d-a82c-74bf1e9bd76f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:56.770830 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 837/1663] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:56 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:12:56.775708 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a1c60-5ebe-4024-b02c-2248b3269dea admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:56.786142 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a1c60-5ebe-4024-b02c-2248b3269dea admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:56.844433 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44efd89-b597-4f0f-bf4d-b480047ca5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:56.845019 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44efd89-b597-4f0f-bf4d-b480047ca5ab admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:56.845328 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44efd89-b597-4f0f-bf4d-b480047ca5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:56.845630 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44efd89-b597-4f0f-bf4d-b480047ca5ab admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:56.846306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44efd89-b597-4f0f-bf4d-b480047ca5ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:56.848386 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44efd89-b597-4f0f-bf4d-b480047ca5ab admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:56.872271 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 827/1664] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:56 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:56.876724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90634610-4073-434d-a82c-74bf1e9bd76f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:56.898007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90634610-4073-434d-a82c-74bf1e9bd76f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:56.903668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44920a9d-8d2b-415e-8c61-5c236cafd558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:56.904205 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44920a9d-8d2b-415e-8c61-5c236cafd558 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:56.904542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44920a9d-8d2b-415e-8c61-5c236cafd558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:56.904755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44920a9d-8d2b-415e-8c61-5c236cafd558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:56.905407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44920a9d-8d2b-415e-8c61-5c236cafd558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:56.907039 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44920a9d-8d2b-415e-8c61-5c236cafd558 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:56.923854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 838/1665] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:56.927837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44efd89-b597-4f0f-bf4d-b480047ca5ab admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:56.938287 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44efd89-b597-4f0f-bf4d-b480047ca5ab admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:56.943211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7032c6-e3e4-4516-9d8f-6ff43f166fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:56.944487 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7032c6-e3e4-4516-9d8f-6ff43f166fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:56.944841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7032c6-e3e4-4516-9d8f-6ff43f166fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:56.945274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7032c6-e3e4-4516-9d8f-6ff43f166fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:56.945932 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7032c6-e3e4-4516-9d8f-6ff43f166fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:56.947611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7032c6-e3e4-4516-9d8f-6ff43f166fe5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:56.960975 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 828/1666] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:56.965333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44920a9d-8d2b-415e-8c61-5c236cafd558 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:56.976334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44920a9d-8d2b-415e-8c61-5c236cafd558 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:56.981411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb5353-34c7-4ec3-964e-5b876f3dcc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:56.982098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb5353-34c7-4ec3-964e-5b876f3dcc80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:56.982374 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb5353-34c7-4ec3-964e-5b876f3dcc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:56.982724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb5353-34c7-4ec3-964e-5b876f3dcc80 admin admin] PATH_INFO: `/v3/projects/3eee962df4ba474caf38b1f33539a8de/users/f9370e05cdd944ce82a9625ebf0c0918/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:56.985004 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:57.002083 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb5353-34c7-4ec3-964e-5b876f3dcc80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eee962df4ba474caf38b1f33539a8de, user_id=f9370e05cdd944ce82a9625ebf0c0918, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:57.004799 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb5353-34c7-4ec3-964e-5b876f3dcc80 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:57.018586 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 839/1667] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:56 2022] PUT /identity/v3/projects/3eee962df4ba474caf38b1f33539a8de/users/f9370e05cdd944ce82a9625ebf0c0918/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:57.030843 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788da7f-bfb7-4d3a-90cf-478ad6aa3453 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:57.031413 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788da7f-bfb7-4d3a-90cf-478ad6aa3453 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:57.031535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788da7f-bfb7-4d3a-90cf-478ad6aa3453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:57.058495 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e788da7f-bfb7-4d3a-90cf-478ad6aa3453 None None] MFA Rules not processed for user `f9370e05cdd944ce82a9625ebf0c0918`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:57.059732 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e788da7f-bfb7-4d3a-90cf-478ad6aa3453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:57.067790 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:57.126431 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 829/1668] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:57.131113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb5353-34c7-4ec3-964e-5b876f3dcc80 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:57.193218 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb5353-34c7-4ec3-964e-5b876f3dcc80 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:57.198644 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd3b6a-880c-4beb-95eb-aa0ba2dd90af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:57.199275 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd3b6a-880c-4beb-95eb-aa0ba2dd90af admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:57.199557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd3b6a-880c-4beb-95eb-aa0ba2dd90af admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:57.199829 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd3b6a-880c-4beb-95eb-aa0ba2dd90af admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:57.200468 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd3b6a-880c-4beb-95eb-aa0ba2dd90af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:57.202357 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd3b6a-880c-4beb-95eb-aa0ba2dd90af admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:57.230495 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 840/1669] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:12:57 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:57.234518 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788da7f-bfb7-4d3a-90cf-478ad6aa3453 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:57.245744 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788da7f-bfb7-4d3a-90cf-478ad6aa3453 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:57.309986 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9006116-4f3f-4062-af4d-17c6c5f64d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:57.310597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9006116-4f3f-4062-af4d-17c6c5f64d6e admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:57.310893 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9006116-4f3f-4062-af4d-17c6c5f64d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:57.311192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9006116-4f3f-4062-af4d-17c6c5f64d6e admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:57.311842 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9006116-4f3f-4062-af4d-17c6c5f64d6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:57.313887 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9006116-4f3f-4062-af4d-17c6c5f64d6e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:57.335923 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 830/1670] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:12:57 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:12:57.340220 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd3b6a-880c-4beb-95eb-aa0ba2dd90af admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:57.351420 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd3b6a-880c-4beb-95eb-aa0ba2dd90af admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:57.355997 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df58da-e19a-4aea-aa67-9167609be477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:57.356623 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df58da-e19a-4aea-aa67-9167609be477 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:57.356886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df58da-e19a-4aea-aa67-9167609be477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:57.357154 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df58da-e19a-4aea-aa67-9167609be477 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:57.357788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df58da-e19a-4aea-aa67-9167609be477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:57.359440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df58da-e19a-4aea-aa67-9167609be477 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:57.369798 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 841/1671] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:57.373883 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9006116-4f3f-4062-af4d-17c6c5f64d6e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:57.383600 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9006116-4f3f-4062-af4d-17c6c5f64d6e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:57.388120 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0776fd3-f1f4-4591-8fa4-d72a6334e4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:57.388759 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0776fd3-f1f4-4591-8fa4-d72a6334e4eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:57.389035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0776fd3-f1f4-4591-8fa4-d72a6334e4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:57.389301 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0776fd3-f1f4-4591-8fa4-d72a6334e4eb admin admin] PATH_INFO: `/v3/projects/6bb487fabcd640beacaa9947b3bfcabe/users/eac1fe02ff90438ea4c99b8e5add8ba9/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:57.404421 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0776fd3-f1f4-4591-8fa4-d72a6334e4eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb487fabcd640beacaa9947b3bfcabe, user_id=eac1fe02ff90438ea4c99b8e5add8ba9, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:57.407095 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0776fd3-f1f4-4591-8fa4-d72a6334e4eb admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:57.420387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 831/1672] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:57 2022] PUT /identity/v3/projects/6bb487fabcd640beacaa9947b3bfcabe/users/eac1fe02ff90438ea4c99b8e5add8ba9/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:57.424261 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df58da-e19a-4aea-aa67-9167609be477 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:57.434125 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df58da-e19a-4aea-aa67-9167609be477 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:57.488933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a646c6-61f0-46db-b8da-a249062845db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:57.489538 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a646c6-61f0-46db-b8da-a249062845db admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:57.489817 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a646c6-61f0-46db-b8da-a249062845db admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:57.490088 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a646c6-61f0-46db-b8da-a249062845db admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:57.490748 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a646c6-61f0-46db-b8da-a249062845db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:57.492358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a646c6-61f0-46db-b8da-a249062845db admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:57.501906 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 842/1673] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:12:57.506046 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0776fd3-f1f4-4591-8fa4-d72a6334e4eb admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:57.516919 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0776fd3-f1f4-4591-8fa4-d72a6334e4eb admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:57.521584 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c9ba2-8694-4203-b81d-4756f8d679d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:57.522542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c9ba2-8694-4203-b81d-4756f8d679d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:57.522827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c9ba2-8694-4203-b81d-4756f8d679d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:57.523135 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c9ba2-8694-4203-b81d-4756f8d679d4 admin admin] PATH_INFO: `/v3/projects/6bb487fabcd640beacaa9947b3bfcabe/users/eac1fe02ff90438ea4c99b8e5add8ba9/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:57.525061 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c9ba2-8694-4203-b81d-4756f8d679d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb487fabcd640beacaa9947b3bfcabe, user_id=eac1fe02ff90438ea4c99b8e5add8ba9, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:57.528075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c9ba2-8694-4203-b81d-4756f8d679d4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:57.538944 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 832/1674] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:12:57 2022] PUT /identity/v3/projects/6bb487fabcd640beacaa9947b3bfcabe/users/eac1fe02ff90438ea4c99b8e5add8ba9/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:57.550319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a953b-3fee-4e2b-9e96-2f6b3709b5b0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:57.550604 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a953b-3fee-4e2b-9e96-2f6b3709b5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:57.550892 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a953b-3fee-4e2b-9e96-2f6b3709b5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:57.581856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3a4a953b-3fee-4e2b-9e96-2f6b3709b5b0 None None] MFA Rules not processed for user `eac1fe02ff90438ea4c99b8e5add8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:57.583128 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3a4a953b-3fee-4e2b-9e96-2f6b3709b5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:57.625403 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:57.654982 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 843/1675] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:57.667460 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c9ba2-8694-4203-b81d-4756f8d679d4 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:57.669211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-820c9ba2-8694-4203-b81d-4756f8d679d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:57.726931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c9ba2-8694-4203-b81d-4756f8d679d4 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:57.730539 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd19b5a-027c-4264-8aa7-a862af379319 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:57.731162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd19b5a-027c-4264-8aa7-a862af379319 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:57.731438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd19b5a-027c-4264-8aa7-a862af379319 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:57.731710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd19b5a-027c-4264-8aa7-a862af379319 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:57.740473 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd19b5a-027c-4264-8aa7-a862af379319 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:57.742985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd19b5a-027c-4264-8aa7-a862af379319 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:57.752244 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 833/1676] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:58.136860 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a953b-3fee-4e2b-9e96-2f6b3709b5b0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:58.218338 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a953b-3fee-4e2b-9e96-2f6b3709b5b0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:58.223033 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846beb9d-2405-46a1-947b-f18b055cf8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:58.223673 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846beb9d-2405-46a1-947b-f18b055cf8fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:58.223958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846beb9d-2405-46a1-947b-f18b055cf8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:58.224265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846beb9d-2405-46a1-947b-f18b055cf8fe admin admin] PATH_INFO: `/v3/users/f9370e05cdd944ce82a9625ebf0c0918` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:58.225972 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846beb9d-2405-46a1-947b-f18b055cf8fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9370e05cdd944ce82a9625ebf0c0918)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:58.227998 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846beb9d-2405-46a1-947b-f18b055cf8fe admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:58.273568 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-846beb9d-2405-46a1-947b-f18b055cf8fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9370e05cdd944ce82a9625ebf0c0918'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:58.281500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-846beb9d-2405-46a1-947b-f18b055cf8fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9370e05cdd944ce82a9625ebf0c0918'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:58.282236 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-846beb9d-2405-46a1-947b-f18b055cf8fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9370e05cdd944ce82a9625ebf0c0918'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:58.289831 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-846beb9d-2405-46a1-947b-f18b055cf8fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9370e05cdd944ce82a9625ebf0c0918'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:58.301571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-846beb9d-2405-46a1-947b-f18b055cf8fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9370e05cdd944ce82a9625ebf0c0918'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:58.307432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 844/1677] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:58 2022] DELETE /identity/v3/users/f9370e05cdd944ce82a9625ebf0c0918 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:58.311863 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e3471-4ef8-4db8-9c27-580c929a8d77 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:58.312321 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e3471-4ef8-4db8-9c27-580c929a8d77 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:58.312409 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e3471-4ef8-4db8-9c27-580c929a8d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:58.321308 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:58.333165 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f27e3471-4ef8-4db8-9c27-580c929a8d77 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:58.334366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f27e3471-4ef8-4db8-9c27-580c929a8d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:58.389379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846beb9d-2405-46a1-947b-f18b055cf8fe admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:58.390795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-846beb9d-2405-46a1-947b-f18b055cf8fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:58.398924 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 834/1678] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:58.406464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e3471-4ef8-4db8-9c27-580c929a8d77 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:58.407873 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f27e3471-4ef8-4db8-9c27-580c929a8d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:58.409668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846beb9d-2405-46a1-947b-f18b055cf8fe admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:58.415018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cac237-43ce-4418-804a-a0ff1b2a273f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:58.415661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cac237-43ce-4418-804a-a0ff1b2a273f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:58.415942 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cac237-43ce-4418-804a-a0ff1b2a273f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:58.416241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cac237-43ce-4418-804a-a0ff1b2a273f admin admin] PATH_INFO: `/v3/users/19fe1c8eb65d477b99e6c79dce57fc51` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:58.417366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cac237-43ce-4418-804a-a0ff1b2a273f admin admin] RBAC: Authorizing `identity:delete_user(user_id=19fe1c8eb65d477b99e6c79dce57fc51)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:58.419364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cac237-43ce-4418-804a-a0ff1b2a273f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:58.466817 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b5cac237-43ce-4418-804a-a0ff1b2a273f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19fe1c8eb65d477b99e6c79dce57fc51'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:58.467910 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e3471-4ef8-4db8-9c27-580c929a8d77 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:58.471508 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff04f9-67b0-4dc6-8e30-220613aa601c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:58.471818 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:58.472120 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff04f9-67b0-4dc6-8e30-220613aa601c service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:58.472593 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff04f9-67b0-4dc6-8e30-220613aa601c service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:58.472708 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff04f9-67b0-4dc6-8e30-220613aa601c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:58.475734 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b5cac237-43ce-4418-804a-a0ff1b2a273f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19fe1c8eb65d477b99e6c79dce57fc51'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:58.476566 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b5cac237-43ce-4418-804a-a0ff1b2a273f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19fe1c8eb65d477b99e6c79dce57fc51'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:58.484094 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff04f9-67b0-4dc6-8e30-220613aa601c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:58.484256 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b5cac237-43ce-4418-804a-a0ff1b2a273f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19fe1c8eb65d477b99e6c79dce57fc51'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:58.485512 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff04f9-67b0-4dc6-8e30-220613aa601c service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:58.496164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 835/1679] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:12:58.498508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b5cac237-43ce-4418-804a-a0ff1b2a273f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19fe1c8eb65d477b99e6c79dce57fc51'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:58.504733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 845/1680] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:58 2022] DELETE /identity/v3/users/19fe1c8eb65d477b99e6c79dce57fc51 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:12:58.509729 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9243e1-9330-42b7-bcb4-43f1e9088c52 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:58.510017 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9243e1-9330-42b7-bcb4-43f1e9088c52 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:58.510450 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9243e1-9330-42b7-bcb4-43f1e9088c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:58.529635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-bb9243e1-9330-42b7-bcb4-43f1e9088c52 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:58.530830 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bb9243e1-9330-42b7-bcb4-43f1e9088c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:58.594105 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 836/1681] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:12:58.604552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cac237-43ce-4418-804a-a0ff1b2a273f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:58.606016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b5cac237-43ce-4418-804a-a0ff1b2a273f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:58.673895 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cac237-43ce-4418-804a-a0ff1b2a273f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:58.679320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27c1aa-1170-4667-9ac3-2fd540f3143d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:58.683633 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27c1aa-1170-4667-9ac3-2fd540f3143d service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:58.683920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27c1aa-1170-4667-9ac3-2fd540f3143d service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:58.684208 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27c1aa-1170-4667-9ac3-2fd540f3143d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:58.697419 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27c1aa-1170-4667-9ac3-2fd540f3143d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:58.699684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27c1aa-1170-4667-9ac3-2fd540f3143d service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:58.712258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 846/1682] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:12:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:12:58.829299 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9243e1-9330-42b7-bcb4-43f1e9088c52 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:58.831387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bb9243e1-9330-42b7-bcb4-43f1e9088c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:58.848134 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9243e1-9330-42b7-bcb4-43f1e9088c52 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:58.855034 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc18f91-719e-435c-a9b7-0214c1d262a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:58.855773 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc18f91-719e-435c-a9b7-0214c1d262a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:58.855773 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc18f91-719e-435c-a9b7-0214c1d262a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:58.856129 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc18f91-719e-435c-a9b7-0214c1d262a6 admin admin] PATH_INFO: `/v3/projects/3eee962df4ba474caf38b1f33539a8de` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:58.857224 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc18f91-719e-435c-a9b7-0214c1d262a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eee962df4ba474caf38b1f33539a8de)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:58.863275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc18f91-719e-435c-a9b7-0214c1d262a6 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:58.922883 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:58.927531 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4fc18f91-719e-435c-a9b7-0214c1d262a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eee962df4ba474caf38b1f33539a8de', 'request_id': 'req-4fc18f91-719e-435c-a9b7-0214c1d262a6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:58.948727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4fc18f91-719e-435c-a9b7-0214c1d262a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eee962df4ba474caf38b1f33539a8de', 'request_id': 'req-4fc18f91-719e-435c-a9b7-0214c1d262a6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:58.960651 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4fc18f91-719e-435c-a9b7-0214c1d262a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eee962df4ba474caf38b1f33539a8de', 'request_id': 'req-4fc18f91-719e-435c-a9b7-0214c1d262a6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:58.976806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4fc18f91-719e-435c-a9b7-0214c1d262a6 admin admin] The token cache is being invalidate because project 3eee962df4ba474caf38b1f33539a8de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:58.977516 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4fc18f91-719e-435c-a9b7-0214c1d262a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:58.977751 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4fc18f91-719e-435c-a9b7-0214c1d262a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:58.979556 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 837/1683] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:58 2022] DELETE /identity/v3/projects/3eee962df4ba474caf38b1f33539a8de => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:58.983553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27c1aa-1170-4667-9ac3-2fd540f3143d service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:58.997716 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27c1aa-1170-4667-9ac3-2fd540f3143d service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:59.076600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7af923-7ea4-4a1f-a4d6-9a5e02a1ff29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:59.077513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7af923-7ea4-4a1f-a4d6-9a5e02a1ff29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:59.077898 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7af923-7ea4-4a1f-a4d6-9a5e02a1ff29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:59.078203 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7af923-7ea4-4a1f-a4d6-9a5e02a1ff29 admin admin] PATH_INFO: `/v3/users/eac1fe02ff90438ea4c99b8e5add8ba9` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:59.079997 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7af923-7ea4-4a1f-a4d6-9a5e02a1ff29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac1fe02ff90438ea4c99b8e5add8ba9)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:59.082476 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7af923-7ea4-4a1f-a4d6-9a5e02a1ff29 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:59.109697 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc18f91-719e-435c-a9b7-0214c1d262a6 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:59.110639 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4fc18f91-719e-435c-a9b7-0214c1d262a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:59.142427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc18f91-719e-435c-a9b7-0214c1d262a6 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:59.147844 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb0690-30f6-4f7f-a262-474424908eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:59.148571 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb0690-30f6-4f7f-a262-474424908eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:59.148846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb0690-30f6-4f7f-a262-474424908eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:59.149408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb0690-30f6-4f7f-a262-474424908eba admin admin] PATH_INFO: `/v3/projects/fe626ac29e53476d8016ee98dea3a9b1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:59.150706 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb0690-30f6-4f7f-a262-474424908eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe626ac29e53476d8016ee98dea3a9b1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:59.154063 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb0690-30f6-4f7f-a262-474424908eba admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:59.157954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2c7af923-7ea4-4a1f-a4d6-9a5e02a1ff29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac1fe02ff90438ea4c99b8e5add8ba9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.168094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2c7af923-7ea4-4a1f-a4d6-9a5e02a1ff29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac1fe02ff90438ea4c99b8e5add8ba9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.169388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2c7af923-7ea4-4a1f-a4d6-9a5e02a1ff29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac1fe02ff90438ea4c99b8e5add8ba9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.180041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2c7af923-7ea4-4a1f-a4d6-9a5e02a1ff29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac1fe02ff90438ea4c99b8e5add8ba9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.193539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:59.203160 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2c7af923-7ea4-4a1f-a4d6-9a5e02a1ff29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac1fe02ff90438ea4c99b8e5add8ba9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.226464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 847/1684] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:58 2022] DELETE /identity/v3/users/eac1fe02ff90438ea4c99b8e5add8ba9 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:59.237969 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-30fb0690-30f6-4f7f-a262-474424908eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe626ac29e53476d8016ee98dea3a9b1', 'request_id': 'req-30fb0690-30f6-4f7f-a262-474424908eba'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.245607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-30fb0690-30f6-4f7f-a262-474424908eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe626ac29e53476d8016ee98dea3a9b1', 'request_id': 'req-30fb0690-30f6-4f7f-a262-474424908eba'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.255263 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-30fb0690-30f6-4f7f-a262-474424908eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe626ac29e53476d8016ee98dea3a9b1', 'request_id': 'req-30fb0690-30f6-4f7f-a262-474424908eba'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.274220 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-30fb0690-30f6-4f7f-a262-474424908eba admin admin] The token cache is being invalidate because project fe626ac29e53476d8016ee98dea3a9b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:59.274220 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-30fb0690-30f6-4f7f-a262-474424908eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.274220 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-30fb0690-30f6-4f7f-a262-474424908eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.274527 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 838/1685] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:59 2022] DELETE /identity/v3/projects/fe626ac29e53476d8016ee98dea3a9b1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:12:59.279019 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7af923-7ea4-4a1f-a4d6-9a5e02a1ff29 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:59.283441 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2c7af923-7ea4-4a1f-a4d6-9a5e02a1ff29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:59.363684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7af923-7ea4-4a1f-a4d6-9a5e02a1ff29 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:59.369287 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d14f1-276e-47eb-99ec-198a8758b3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:59.370027 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d14f1-276e-47eb-99ec-198a8758b3d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:59.370590 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d14f1-276e-47eb-99ec-198a8758b3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:59.370704 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d14f1-276e-47eb-99ec-198a8758b3d0 admin admin] PATH_INFO: `/v3/users/dec10d9dd14d44739f7bd710d29da065` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:59.371907 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d14f1-276e-47eb-99ec-198a8758b3d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dec10d9dd14d44739f7bd710d29da065)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:59.373885 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d14f1-276e-47eb-99ec-198a8758b3d0 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:59.454528 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0d0d14f1-276e-47eb-99ec-198a8758b3d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dec10d9dd14d44739f7bd710d29da065'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.462582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0d0d14f1-276e-47eb-99ec-198a8758b3d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dec10d9dd14d44739f7bd710d29da065'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.463416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0d0d14f1-276e-47eb-99ec-198a8758b3d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dec10d9dd14d44739f7bd710d29da065'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.471242 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0d0d14f1-276e-47eb-99ec-198a8758b3d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dec10d9dd14d44739f7bd710d29da065'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.484806 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0d0d14f1-276e-47eb-99ec-198a8758b3d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dec10d9dd14d44739f7bd710d29da065'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.492750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 848/1686] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:59 2022] DELETE /identity/v3/users/dec10d9dd14d44739f7bd710d29da065 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:59.647689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb0690-30f6-4f7f-a262-474424908eba admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:59.649732 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-30fb0690-30f6-4f7f-a262-474424908eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:59.734263 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb0690-30f6-4f7f-a262-474424908eba admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:59.792155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d14f1-276e-47eb-99ec-198a8758b3d0 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:59.793596 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0d0d14f1-276e-47eb-99ec-198a8758b3d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:59.798689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92640a-bc5e-49ea-b4e1-d5b9a21f7a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:59.800939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92640a-bc5e-49ea-b4e1-d5b9a21f7a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:59.800939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92640a-bc5e-49ea-b4e1-d5b9a21f7a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:59.800939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92640a-bc5e-49ea-b4e1-d5b9a21f7a02 admin admin] PATH_INFO: `/v3/projects/6bb487fabcd640beacaa9947b3bfcabe` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:59.801551 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92640a-bc5e-49ea-b4e1-d5b9a21f7a02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bb487fabcd640beacaa9947b3bfcabe)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:59.804078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92640a-bc5e-49ea-b4e1-d5b9a21f7a02 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:59.806831 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:59.811621 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d14f1-276e-47eb-99ec-198a8758b3d0 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:59.817939 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991927ac-7721-4d83-a6e2-9df4e127ec8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:59.819329 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991927ac-7721-4d83-a6e2-9df4e127ec8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:59.819628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991927ac-7721-4d83-a6e2-9df4e127ec8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:59.819899 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991927ac-7721-4d83-a6e2-9df4e127ec8b admin admin] PATH_INFO: `/v3/projects/a7bbf740783e495981ad50749641174c` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:59.821069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991927ac-7721-4d83-a6e2-9df4e127ec8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7bbf740783e495981ad50749641174c)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:59.823039 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991927ac-7721-4d83-a6e2-9df4e127ec8b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:59.862372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bb92640a-bc5e-49ea-b4e1-d5b9a21f7a02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bb487fabcd640beacaa9947b3bfcabe', 'request_id': 'req-bb92640a-bc5e-49ea-b4e1-d5b9a21f7a02'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.868819 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bb92640a-bc5e-49ea-b4e1-d5b9a21f7a02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bb487fabcd640beacaa9947b3bfcabe', 'request_id': 'req-bb92640a-bc5e-49ea-b4e1-d5b9a21f7a02'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.872781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-991927ac-7721-4d83-a6e2-9df4e127ec8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7bbf740783e495981ad50749641174c', 'request_id': 'req-991927ac-7721-4d83-a6e2-9df4e127ec8b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.875431 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bb92640a-bc5e-49ea-b4e1-d5b9a21f7a02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bb487fabcd640beacaa9947b3bfcabe', 'request_id': 'req-bb92640a-bc5e-49ea-b4e1-d5b9a21f7a02'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.878388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-991927ac-7721-4d83-a6e2-9df4e127ec8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7bbf740783e495981ad50749641174c', 'request_id': 'req-991927ac-7721-4d83-a6e2-9df4e127ec8b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.886655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-991927ac-7721-4d83-a6e2-9df4e127ec8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7bbf740783e495981ad50749641174c', 'request_id': 'req-991927ac-7721-4d83-a6e2-9df4e127ec8b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.888981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bb92640a-bc5e-49ea-b4e1-d5b9a21f7a02 admin admin] The token cache is being invalidate because project 6bb487fabcd640beacaa9947b3bfcabe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:59.889272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bb92640a-bc5e-49ea-b4e1-d5b9a21f7a02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.889877 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:12:59.890024 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bb92640a-bc5e-49ea-b4e1-d5b9a21f7a02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.891507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 839/1687] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:59 2022] DELETE /identity/v3/projects/6bb487fabcd640beacaa9947b3bfcabe => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:59.896531 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41698ae3-64e0-4979-9e7f-81df71bc390e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:59.896989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41698ae3-64e0-4979-9e7f-81df71bc390e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:59.897513 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41698ae3-64e0-4979-9e7f-81df71bc390e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:59.906188 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-991927ac-7721-4d83-a6e2-9df4e127ec8b admin admin] The token cache is being invalidate because project a7bbf740783e495981ad50749641174c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:12:59.906417 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-991927ac-7721-4d83-a6e2-9df4e127ec8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.907111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-991927ac-7721-4d83-a6e2-9df4e127ec8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:12:59.908617 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 849/1688] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:12:59 2022] DELETE /identity/v3/projects/a7bbf740783e495981ad50749641174c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:12:59.912960 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991927ac-7721-4d83-a6e2-9df4e127ec8b admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:12:59.914491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-991927ac-7721-4d83-a6e2-9df4e127ec8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:59.920283 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-41698ae3-64e0-4979-9e7f-81df71bc390e None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:12:59.921957 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-41698ae3-64e0-4979-9e7f-81df71bc390e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:12:59.981700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991927ac-7721-4d83-a6e2-9df4e127ec8b admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:12:59.985564 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 840/1689] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:12:59.986498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6bcaf-9965-42ea-a8a6-a2b076437293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:12:59.987171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6bcaf-9965-42ea-a8a6-a2b076437293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:59.987457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6bcaf-9965-42ea-a8a6-a2b076437293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:59.987734 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6bcaf-9965-42ea-a8a6-a2b076437293 admin admin] PATH_INFO: `/v3/users/6d2bc5750b00497aabe85a022c945eb5` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:12:59.988884 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6bcaf-9965-42ea-a8a6-a2b076437293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d2bc5750b00497aabe85a022c945eb5)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:12:59.991263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6bcaf-9965-42ea-a8a6-a2b076437293 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:12:59.993784 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b5a8c-b00c-4612-bab0-4dcaf7080d79 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:12:59.994186 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b5a8c-b00c-4612-bab0-4dcaf7080d79 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:12:59.994470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b5a8c-b00c-4612-bab0-4dcaf7080d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:00.022174 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-758b5a8c-b00c-4612-bab0-4dcaf7080d79 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:00.023855 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-758b5a8c-b00c-4612-bab0-4dcaf7080d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:00.034696 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 841/1690] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:13:00.039526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b5a8c-b00c-4612-bab0-4dcaf7080d79 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:00.045692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a8c6bcaf-9965-42ea-a8a6-a2b076437293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d2bc5750b00497aabe85a022c945eb5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:00.050526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b5a8c-b00c-4612-bab0-4dcaf7080d79 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:00.054371 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a8c6bcaf-9965-42ea-a8a6-a2b076437293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d2bc5750b00497aabe85a022c945eb5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:00.055189 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a8c6bcaf-9965-42ea-a8a6-a2b076437293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d2bc5750b00497aabe85a022c945eb5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:00.057103 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb7e58-eb56-4051-bfbc-04333762b2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:00.057901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb7e58-eb56-4051-bfbc-04333762b2ca admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:00.058257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb7e58-eb56-4051-bfbc-04333762b2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:00.058604 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb7e58-eb56-4051-bfbc-04333762b2ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:00.059457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb7e58-eb56-4051-bfbc-04333762b2ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:00.059764 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb7e58-eb56-4051-bfbc-04333762b2ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:00.061553 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb7e58-eb56-4051-bfbc-04333762b2ca admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:00.063798 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a8c6bcaf-9965-42ea-a8a6-a2b076437293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d2bc5750b00497aabe85a022c945eb5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:00.077486 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 842/1691] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:13:00.082856 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb7e58-eb56-4051-bfbc-04333762b2ca admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:00.085068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a8c6bcaf-9965-42ea-a8a6-a2b076437293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d2bc5750b00497aabe85a022c945eb5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:00.085953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1fdb7e58-eb56-4051-bfbc-04333762b2ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:00.097653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 850/1692] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:12:59 2022] DELETE /identity/v3/users/6d2bc5750b00497aabe85a022c945eb5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:13:00.103912 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb7e58-eb56-4051-bfbc-04333762b2ca admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:00.174368 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ac1fa-1bed-48f5-8c40-e280c00cfc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:00.174988 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ac1fa-1bed-48f5-8c40-e280c00cfc39 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:00.175352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ac1fa-1bed-48f5-8c40-e280c00cfc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:00.175638 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ac1fa-1bed-48f5-8c40-e280c00cfc39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:00.176292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ac1fa-1bed-48f5-8c40-e280c00cfc39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:00.179877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ac1fa-1bed-48f5-8c40-e280c00cfc39 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:00.207200 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:00.223053 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 843/1693] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:00 2022] POST /identity/v3/projects => generated 373 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:00.226497 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6bcaf-9965-42ea-a8a6-a2b076437293 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:00.237252 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6bcaf-9965-42ea-a8a6-a2b076437293 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:00.312273 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9709c-a343-4ff1-8901-fd6e99b5adf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:00.312903 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9709c-a343-4ff1-8901-fd6e99b5adf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:00.313187 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9709c-a343-4ff1-8901-fd6e99b5adf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:00.313460 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9709c-a343-4ff1-8901-fd6e99b5adf1 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:00.314362 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9709c-a343-4ff1-8901-fd6e99b5adf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:00.316803 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9709c-a343-4ff1-8901-fd6e99b5adf1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:00.342898 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 851/1694] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:00 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:00.346654 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ac1fa-1bed-48f5-8c40-e280c00cfc39 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:00.357631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ac1fa-1bed-48f5-8c40-e280c00cfc39 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:00.362315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed6a8d-c5f0-451c-967d-16240bdefee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:00.362961 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed6a8d-c5f0-451c-967d-16240bdefee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:00.363246 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed6a8d-c5f0-451c-967d-16240bdefee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:00.363521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed6a8d-c5f0-451c-967d-16240bdefee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:00.364168 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed6a8d-c5f0-451c-967d-16240bdefee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:00.366672 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed6a8d-c5f0-451c-967d-16240bdefee1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:00.376601 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 844/1695] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:00.380902 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9709c-a343-4ff1-8901-fd6e99b5adf1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:00.387072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:00.393514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9709c-a343-4ff1-8901-fd6e99b5adf1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:00.400907 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6832703-bfe0-4cac-a2ee-3b3637ed8ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:00.401692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6832703-bfe0-4cac-a2ee-3b3637ed8ffb admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:00.402128 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6832703-bfe0-4cac-a2ee-3b3637ed8ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:00.402815 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6832703-bfe0-4cac-a2ee-3b3637ed8ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:00.403660 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6832703-bfe0-4cac-a2ee-3b3637ed8ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:00.405473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6832703-bfe0-4cac-a2ee-3b3637ed8ffb admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:00.417131 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 852/1696] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:00.422681 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6832703-bfe0-4cac-a2ee-3b3637ed8ffb admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:00.425260 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed6a8d-c5f0-451c-967d-16240bdefee1 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:00.426563 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f3ed6a8d-c5f0-451c-967d-16240bdefee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:00.436523 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6832703-bfe0-4cac-a2ee-3b3637ed8ffb admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:00.441722 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a36ba60-bf4d-41ee-8f71-b44ae1193af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:00.441906 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed6a8d-c5f0-451c-967d-16240bdefee1 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:00.443088 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36ba60-bf4d-41ee-8f71-b44ae1193af4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:00.443454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36ba60-bf4d-41ee-8f71-b44ae1193af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:00.444153 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36ba60-bf4d-41ee-8f71-b44ae1193af4 admin admin] PATH_INFO: `/v3/projects/513e86ab3af74915bbbbc7dc52779c2a/users/8ceb4963c84c4049becaa196680721d3/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:00.450856 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6808922-5b6d-4d48-979a-7b83e024d8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:00.450856 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6808922-5b6d-4d48-979a-7b83e024d8da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:00.450856 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6808922-5b6d-4d48-979a-7b83e024d8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:00.450856 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6808922-5b6d-4d48-979a-7b83e024d8da admin admin] PATH_INFO: `/v3/projects/74deef577f8e47b19776fa862ae13339` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:00.452328 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6808922-5b6d-4d48-979a-7b83e024d8da admin admin] RBAC: Authorizing `identity:delete_project(project_id=74deef577f8e47b19776fa862ae13339)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:00.454268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6808922-5b6d-4d48-979a-7b83e024d8da admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:00.465291 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a36ba60-bf4d-41ee-8f71-b44ae1193af4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=513e86ab3af74915bbbbc7dc52779c2a, user_id=8ceb4963c84c4049becaa196680721d3, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:00.469014 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a36ba60-bf4d-41ee-8f71-b44ae1193af4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:00.488907 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 853/1697] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:00 2022] PUT /identity/v3/projects/513e86ab3af74915bbbbc7dc52779c2a/users/8ceb4963c84c4049becaa196680721d3/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:00.503659 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be71e2-7c18-422f-a9dd-ca43a485f1da None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:00.504110 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be71e2-7c18-422f-a9dd-ca43a485f1da None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:00.504562 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be71e2-7c18-422f-a9dd-ca43a485f1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:00.536054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a6808922-5b6d-4d48-979a-7b83e024d8da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74deef577f8e47b19776fa862ae13339', 'request_id': 'req-a6808922-5b6d-4d48-979a-7b83e024d8da'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:00.538642 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-03be71e2-7c18-422f-a9dd-ca43a485f1da None None] MFA Rules not processed for user `8ceb4963c84c4049becaa196680721d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:00.540173 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-03be71e2-7c18-422f-a9dd-ca43a485f1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:00.546832 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a6808922-5b6d-4d48-979a-7b83e024d8da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74deef577f8e47b19776fa862ae13339', 'request_id': 'req-a6808922-5b6d-4d48-979a-7b83e024d8da'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:00.556090 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a6808922-5b6d-4d48-979a-7b83e024d8da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74deef577f8e47b19776fa862ae13339', 'request_id': 'req-a6808922-5b6d-4d48-979a-7b83e024d8da'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:00.580579 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a6808922-5b6d-4d48-979a-7b83e024d8da admin admin] The token cache is being invalidate because project 74deef577f8e47b19776fa862ae13339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:00.581007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a6808922-5b6d-4d48-979a-7b83e024d8da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:00.581772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a6808922-5b6d-4d48-979a-7b83e024d8da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:00.583630 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 845/1698] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:00 2022] DELETE /identity/v3/projects/74deef577f8e47b19776fa862ae13339 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:00.589172 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81d1e5-c38e-42a5-988c-e2017a0edc2f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:00.589482 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81d1e5-c38e-42a5-988c-e2017a0edc2f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:00.589767 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81d1e5-c38e-42a5-988c-e2017a0edc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:00.614831 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-dc81d1e5-c38e-42a5-988c-e2017a0edc2f None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:00.616246 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-dc81d1e5-c38e-42a5-988c-e2017a0edc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:00.654108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 854/1699] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:13:00.677767 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be71e2-7c18-422f-a9dd-ca43a485f1da None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:00.679514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-03be71e2-7c18-422f-a9dd-ca43a485f1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:00.690519 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 846/1700] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:13:00.698985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f544a0-25f7-4e8c-bc88-3ee1de14973f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:00.699304 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f544a0-25f7-4e8c-bc88-3ee1de14973f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:00.699605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f544a0-25f7-4e8c-bc88-3ee1de14973f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:00.719494 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e0f544a0-25f7-4e8c-bc88-3ee1de14973f None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:00.720739 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e0f544a0-25f7-4e8c-bc88-3ee1de14973f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:00.728995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 847/1701] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:13:00.733185 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f544a0-25f7-4e8c-bc88-3ee1de14973f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:00.741286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be71e2-7c18-422f-a9dd-ca43a485f1da None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:00.743955 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f544a0-25f7-4e8c-bc88-3ee1de14973f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:00.745612 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90db166-35a9-4ce9-96b0-b2efad1f3e5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:00.746258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90db166-35a9-4ce9-96b0-b2efad1f3e5d service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:00.746537 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90db166-35a9-4ce9-96b0-b2efad1f3e5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:00.746809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90db166-35a9-4ce9-96b0-b2efad1f3e5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:00.748686 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b7943-f870-48e0-84c0-ee85a2c5a72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:00.749280 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b7943-f870-48e0-84c0-ee85a2c5a72c admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:00.749572 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b7943-f870-48e0-84c0-ee85a2c5a72c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:00.749855 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b7943-f870-48e0-84c0-ee85a2c5a72c admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:00.750819 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b7943-f870-48e0-84c0-ee85a2c5a72c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:00.750957 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b7943-f870-48e0-84c0-ee85a2c5a72c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:00.752680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b7943-f870-48e0-84c0-ee85a2c5a72c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:00.755365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90db166-35a9-4ce9-96b0-b2efad1f3e5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:00.757311 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90db166-35a9-4ce9-96b0-b2efad1f3e5d service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:00.765587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 848/1702] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:13:00.766795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 855/1703] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:13:00.770278 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b7943-f870-48e0-84c0-ee85a2c5a72c admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:00.780869 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b7943-f870-48e0-84c0-ee85a2c5a72c admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:00.785458 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb505a-b2f8-47f0-846a-5b1293abfd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:00.786047 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb505a-b2f8-47f0-846a-5b1293abfd8b admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:00.786327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb505a-b2f8-47f0-846a-5b1293abfd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:00.786597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb505a-b2f8-47f0-846a-5b1293abfd8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:00.787251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb505a-b2f8-47f0-846a-5b1293abfd8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:00.789387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb505a-b2f8-47f0-846a-5b1293abfd8b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:00.790722 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90db166-35a9-4ce9-96b0-b2efad1f3e5d service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:00.792270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a90db166-35a9-4ce9-96b0-b2efad1f3e5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:00.806617 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90db166-35a9-4ce9-96b0-b2efad1f3e5d service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:00.812250 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e3ca3-9a3c-4313-9a76-4d14dd5d37d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:00.813037 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e3ca3-9a3c-4313-9a76-4d14dd5d37d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:00.813349 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e3ca3-9a3c-4313-9a76-4d14dd5d37d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:00.813624 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e3ca3-9a3c-4313-9a76-4d14dd5d37d1 admin admin] PATH_INFO: `/v3/users/8ceb4963c84c4049becaa196680721d3` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:00.815078 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e3ca3-9a3c-4313-9a76-4d14dd5d37d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ceb4963c84c4049becaa196680721d3)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:00.817227 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e3ca3-9a3c-4313-9a76-4d14dd5d37d1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:00.818577 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 849/1704] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:00 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:13:00.823425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb505a-b2f8-47f0-846a-5b1293abfd8b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:00.856536 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:00.874432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9b9e3ca3-9a3c-4313-9a76-4d14dd5d37d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ceb4963c84c4049becaa196680721d3'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:00.880766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:00.883354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9b9e3ca3-9a3c-4313-9a76-4d14dd5d37d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ceb4963c84c4049becaa196680721d3'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:00.884134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9b9e3ca3-9a3c-4313-9a76-4d14dd5d37d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ceb4963c84c4049becaa196680721d3'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:00.894771 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb505a-b2f8-47f0-846a-5b1293abfd8b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:00.894988 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9b9e3ca3-9a3c-4313-9a76-4d14dd5d37d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ceb4963c84c4049becaa196680721d3'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:00.900776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf095a-2bd8-445e-951a-cac459892c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:00.901389 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf095a-2bd8-445e-951a-cac459892c62 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:00.901666 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf095a-2bd8-445e-951a-cac459892c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:00.901937 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf095a-2bd8-445e-951a-cac459892c62 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:00.902836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf095a-2bd8-445e-951a-cac459892c62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:00.904772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf095a-2bd8-445e-951a-cac459892c62 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:00.909729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-9b9e3ca3-9a3c-4313-9a76-4d14dd5d37d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ceb4963c84c4049becaa196680721d3'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:00.918775 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 856/1705] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:00 2022] DELETE /identity/v3/users/8ceb4963c84c4049becaa196680721d3 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:13:00.924080 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b7fcc-c41f-4f0a-b33c-f906f37b46f8 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:00.924373 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b7fcc-c41f-4f0a-b33c-f906f37b46f8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:00.924651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b7fcc-c41f-4f0a-b33c-f906f37b46f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:00.927172 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 850/1706] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:00 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:13:00.931212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf095a-2bd8-445e-951a-cac459892c62 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:00.932816 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-76cf095a-2bd8-445e-951a-cac459892c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:00.944504 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf095a-2bd8-445e-951a-cac459892c62 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:00.944667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0d7b7fcc-c41f-4f0a-b33c-f906f37b46f8 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:00.945700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0d7b7fcc-c41f-4f0a-b33c-f906f37b46f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:01.007801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b8cda-f3ad-49a9-be9f-920343216512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:01.009266 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b8cda-f3ad-49a9-be9f-920343216512 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:01.009768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b8cda-f3ad-49a9-be9f-920343216512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:01.010261 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b8cda-f3ad-49a9-be9f-920343216512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:01.011453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b8cda-f3ad-49a9-be9f-920343216512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:01.014037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b8cda-f3ad-49a9-be9f-920343216512 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:01.015283 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 857/1707] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:01.024978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b7fcc-c41f-4f0a-b33c-f906f37b46f8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:01.026358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0d7b7fcc-c41f-4f0a-b33c-f906f37b46f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:01.030512 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 851/1708] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:13:01.031549 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b8cda-f3ad-49a9-be9f-920343216512 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:01.041345 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b8cda-f3ad-49a9-be9f-920343216512 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:01.045994 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35cf78-ae96-4a32-86b3-613e638eeae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:01.046578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35cf78-ae96-4a32-86b3-613e638eeae7 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:01.046859 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35cf78-ae96-4a32-86b3-613e638eeae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:01.047168 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35cf78-ae96-4a32-86b3-613e638eeae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:01.047806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35cf78-ae96-4a32-86b3-613e638eeae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:01.049397 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35cf78-ae96-4a32-86b3-613e638eeae7 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:01.060941 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 852/1709] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:01.064979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35cf78-ae96-4a32-86b3-613e638eeae7 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:01.075446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35cf78-ae96-4a32-86b3-613e638eeae7 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:01.080060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045e92f-92ea-4c4f-8388-fd723565ba3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:01.080723 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045e92f-92ea-4c4f-8388-fd723565ba3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:01.081001 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045e92f-92ea-4c4f-8388-fd723565ba3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:01.081748 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045e92f-92ea-4c4f-8388-fd723565ba3f admin admin] PATH_INFO: `/v3/projects/b87c159313f74c77a95172124e82dc25/users/c58fbb2d79954020aa1328ef6bf925e4/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:01.085248 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b7fcc-c41f-4f0a-b33c-f906f37b46f8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:01.088972 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ba329-ac9d-4b2f-b754-7d3b7b6efcad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:01.089588 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ba329-ac9d-4b2f-b754-7d3b7b6efcad service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:01.089869 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ba329-ac9d-4b2f-b754-7d3b7b6efcad service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:01.090149 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ba329-ac9d-4b2f-b754-7d3b7b6efcad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:01.096680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045e92f-92ea-4c4f-8388-fd723565ba3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b87c159313f74c77a95172124e82dc25, user_id=c58fbb2d79954020aa1328ef6bf925e4, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:01.099372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b045e92f-92ea-4c4f-8388-fd723565ba3f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:01.099486 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ba329-ac9d-4b2f-b754-7d3b7b6efcad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:01.101054 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ba329-ac9d-4b2f-b754-7d3b7b6efcad service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:01.111246 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 858/1710] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:13:01.112041 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 853/1711] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:01 2022] PUT /identity/v3/projects/b87c159313f74c77a95172124e82dc25/users/c58fbb2d79954020aa1328ef6bf925e4/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:13:01.117604 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346137f0-16f1-411e-85c1-204ab273fe20 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:01.117896 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346137f0-16f1-411e-85c1-204ab273fe20 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:01.118201 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346137f0-16f1-411e-85c1-204ab273fe20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:01.137414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-346137f0-16f1-411e-85c1-204ab273fe20 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:01.138622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-346137f0-16f1-411e-85c1-204ab273fe20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:01.208479 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 859/1712] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:13:01.218976 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045e92f-92ea-4c4f-8388-fd723565ba3f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:01.221321 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b045e92f-92ea-4c4f-8388-fd723565ba3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:01.300984 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045e92f-92ea-4c4f-8388-fd723565ba3f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:01.313258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8182f0e-79c3-4cbc-b8da-4dbcbcd65253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:01.313258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8182f0e-79c3-4cbc-b8da-4dbcbcd65253 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:01.313258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8182f0e-79c3-4cbc-b8da-4dbcbcd65253 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:01.313258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8182f0e-79c3-4cbc-b8da-4dbcbcd65253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:01.327534 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8182f0e-79c3-4cbc-b8da-4dbcbcd65253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:01.329394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8182f0e-79c3-4cbc-b8da-4dbcbcd65253 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:01.347066 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 854/1713] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:01.402970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346137f0-16f1-411e-85c1-204ab273fe20 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:01.404404 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-346137f0-16f1-411e-85c1-204ab273fe20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:01.419351 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346137f0-16f1-411e-85c1-204ab273fe20 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:01.425079 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b2281-c582-4562-8b45-326a763ebe62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:01.425846 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b2281-c582-4562-8b45-326a763ebe62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:01.426184 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b2281-c582-4562-8b45-326a763ebe62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:01.426492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b2281-c582-4562-8b45-326a763ebe62 admin admin] PATH_INFO: `/v3/projects/513e86ab3af74915bbbbc7dc52779c2a` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:01.427913 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b2281-c582-4562-8b45-326a763ebe62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=513e86ab3af74915bbbbc7dc52779c2a)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:01.430952 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b2281-c582-4562-8b45-326a763ebe62 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:01.445296 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:01.504439 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7d9b2281-c582-4562-8b45-326a763ebe62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '513e86ab3af74915bbbbc7dc52779c2a', 'request_id': 'req-7d9b2281-c582-4562-8b45-326a763ebe62'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:01.512400 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7d9b2281-c582-4562-8b45-326a763ebe62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '513e86ab3af74915bbbbc7dc52779c2a', 'request_id': 'req-7d9b2281-c582-4562-8b45-326a763ebe62'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:01.520919 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7d9b2281-c582-4562-8b45-326a763ebe62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '513e86ab3af74915bbbbc7dc52779c2a', 'request_id': 'req-7d9b2281-c582-4562-8b45-326a763ebe62'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:01.538882 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7d9b2281-c582-4562-8b45-326a763ebe62 admin admin] The token cache is being invalidate because project 513e86ab3af74915bbbbc7dc52779c2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:01.539377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7d9b2281-c582-4562-8b45-326a763ebe62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:01.540771 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7d9b2281-c582-4562-8b45-326a763ebe62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:01.547166 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 860/1714] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:01 2022] DELETE /identity/v3/projects/513e86ab3af74915bbbbc7dc52779c2a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:01.559555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b4f71-92a1-4fe0-b92d-8b4f4f5919bb None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:01.561870 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b4f71-92a1-4fe0-b92d-8b4f4f5919bb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:01.562192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b4f71-92a1-4fe0-b92d-8b4f4f5919bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:01.583476 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-545b4f71-92a1-4fe0-b92d-8b4f4f5919bb None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:01.585317 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-545b4f71-92a1-4fe0-b92d-8b4f4f5919bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:01.604250 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:01.653742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 855/1715] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:01.663464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feceda50-7284-4c7f-9f44-f82149ace141 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:01.663762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feceda50-7284-4c7f-9f44-f82149ace141 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:01.664791 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feceda50-7284-4c7f-9f44-f82149ace141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:01.672149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b4f71-92a1-4fe0-b92d-8b4f4f5919bb None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:01.673788 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-545b4f71-92a1-4fe0-b92d-8b4f4f5919bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:01.688393 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b4f71-92a1-4fe0-b92d-8b4f4f5919bb None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:01.696492 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7391de-87b6-4efa-94b8-f0c617b23ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:01.697321 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7391de-87b6-4efa-94b8-f0c617b23ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:01.697705 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7391de-87b6-4efa-94b8-f0c617b23ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:01.698079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7391de-87b6-4efa-94b8-f0c617b23ccf admin admin] PATH_INFO: `/v3/users/675821fcad094531a09e620ac4594e7e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:01.699771 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7391de-87b6-4efa-94b8-f0c617b23ccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=675821fcad094531a09e620ac4594e7e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:01.704268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7391de-87b6-4efa-94b8-f0c617b23ccf admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:01.725362 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-feceda50-7284-4c7f-9f44-f82149ace141 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:01.731777 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-feceda50-7284-4c7f-9f44-f82149ace141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:01.741942 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 861/1716] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:01.746663 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feceda50-7284-4c7f-9f44-f82149ace141 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:01.760086 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feceda50-7284-4c7f-9f44-f82149ace141 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:01.770128 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53165764-d9f3-473a-9bdf-eecc0df0d848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:01.770750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53165764-d9f3-473a-9bdf-eecc0df0d848 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:01.771055 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53165764-d9f3-473a-9bdf-eecc0df0d848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:01.771331 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53165764-d9f3-473a-9bdf-eecc0df0d848 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:01.772514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53165764-d9f3-473a-9bdf-eecc0df0d848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:01.772815 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53165764-d9f3-473a-9bdf-eecc0df0d848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:01.776985 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53165764-d9f3-473a-9bdf-eecc0df0d848 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:01.781821 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5b7391de-87b6-4efa-94b8-f0c617b23ccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '675821fcad094531a09e620ac4594e7e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:01.794267 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5b7391de-87b6-4efa-94b8-f0c617b23ccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '675821fcad094531a09e620ac4594e7e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:01.795318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5b7391de-87b6-4efa-94b8-f0c617b23ccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '675821fcad094531a09e620ac4594e7e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:01.807855 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5b7391de-87b6-4efa-94b8-f0c617b23ccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '675821fcad094531a09e620ac4594e7e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:01.808803 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 862/1717] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:13:01.809761 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53165764-d9f3-473a-9bdf-eecc0df0d848 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:01.811693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-53165764-d9f3-473a-9bdf-eecc0df0d848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:01.824839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5b7391de-87b6-4efa-94b8-f0c617b23ccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '675821fcad094531a09e620ac4594e7e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:01.835760 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 856/1718] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:01 2022] DELETE /identity/v3/users/675821fcad094531a09e620ac4594e7e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:01.843084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53165764-d9f3-473a-9bdf-eecc0df0d848 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:01.849328 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c09a8-8f2b-48c7-8c77-ad5d18ad6ecb None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:01.849592 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c09a8-8f2b-48c7-8c77-ad5d18ad6ecb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:01.849976 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c09a8-8f2b-48c7-8c77-ad5d18ad6ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:01.879438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e20c09a8-8f2b-48c7-8c77-ad5d18ad6ecb None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:01.880793 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e20c09a8-8f2b-48c7-8c77-ad5d18ad6ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:01.943817 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb066f5a-947c-44df-8c66-ce2f22b6e544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:01.944301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb066f5a-947c-44df-8c66-ce2f22b6e544 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:01.944585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb066f5a-947c-44df-8c66-ce2f22b6e544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:01.944854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb066f5a-947c-44df-8c66-ce2f22b6e544 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:01.945547 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb066f5a-947c-44df-8c66-ce2f22b6e544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:01.947833 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb066f5a-947c-44df-8c66-ce2f22b6e544 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:01.973717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 857/1719] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:01.984158 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 863/1720] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:01 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:01.987577 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c09a8-8f2b-48c7-8c77-ad5d18ad6ecb None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:01.987957 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb066f5a-947c-44df-8c66-ce2f22b6e544 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:01.989121 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e20c09a8-8f2b-48c7-8c77-ad5d18ad6ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:02.037743 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:02.061128 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c09a8-8f2b-48c7-8c77-ad5d18ad6ecb None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:02.075546 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb066f5a-947c-44df-8c66-ce2f22b6e544 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:02.080611 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb72009-2e27-416b-873c-d8844ec7f2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:02.081370 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb72009-2e27-416b-873c-d8844ec7f2e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:02.082009 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb72009-2e27-416b-873c-d8844ec7f2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:02.082375 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb72009-2e27-416b-873c-d8844ec7f2e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:02.083232 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb72009-2e27-416b-873c-d8844ec7f2e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:02.086307 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb72009-2e27-416b-873c-d8844ec7f2e9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:02.097824 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:02.117174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 864/1721] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:01 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:13:02.129656 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb72009-2e27-416b-873c-d8844ec7f2e9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:02.136458 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3e58c-da94-4859-bcb2-5bf95290d625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:02.137579 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3e58c-da94-4859-bcb2-5bf95290d625 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:02.137579 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3e58c-da94-4859-bcb2-5bf95290d625 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:02.137801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3e58c-da94-4859-bcb2-5bf95290d625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:02.141338 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb72009-2e27-416b-873c-d8844ec7f2e9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:02.146275 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97807005-42a4-4c7b-95d1-f3869beb5664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:02.147115 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97807005-42a4-4c7b-95d1-f3869beb5664 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:02.147475 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97807005-42a4-4c7b-95d1-f3869beb5664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:02.147822 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97807005-42a4-4c7b-95d1-f3869beb5664 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:02.148544 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97807005-42a4-4c7b-95d1-f3869beb5664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:02.149411 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3e58c-da94-4859-bcb2-5bf95290d625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:02.150621 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97807005-42a4-4c7b-95d1-f3869beb5664 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:02.150891 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3e58c-da94-4859-bcb2-5bf95290d625 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:02.164919 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 865/1722] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:13:02.164919 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a356bb-d9cb-45d1-9c05-f6c12752b57f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:02.164919 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a356bb-d9cb-45d1-9c05-f6c12752b57f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:02.164919 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a356bb-d9cb-45d1-9c05-f6c12752b57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:02.165342 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 858/1723] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:13:02.166575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3e58c-da94-4859-bcb2-5bf95290d625 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:02.180077 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3e58c-da94-4859-bcb2-5bf95290d625 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:02.186899 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a74ea-8e4c-481f-aead-bc198a3da011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:02.188217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a74ea-8e4c-481f-aead-bc198a3da011 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:02.188217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a74ea-8e4c-481f-aead-bc198a3da011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:02.188217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a74ea-8e4c-481f-aead-bc198a3da011 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:02.188955 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a74ea-8e4c-481f-aead-bc198a3da011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:02.190858 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a74ea-8e4c-481f-aead-bc198a3da011 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:02.197749 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-44a356bb-d9cb-45d1-9c05-f6c12752b57f None None] MFA Rules not processed for user `c58fbb2d79954020aa1328ef6bf925e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:02.198935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-44a356bb-d9cb-45d1-9c05-f6c12752b57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:02.210853 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 859/1724] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:13:02.218353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a74ea-8e4c-481f-aead-bc198a3da011 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:02.233393 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a74ea-8e4c-481f-aead-bc198a3da011 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:02.238786 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72d62d-c5e6-4811-a54e-e8ce07eb25bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:02.239615 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72d62d-c5e6-4811-a54e-e8ce07eb25bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:02.240050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72d62d-c5e6-4811-a54e-e8ce07eb25bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:02.240469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72d62d-c5e6-4811-a54e-e8ce07eb25bf admin admin] PATH_INFO: `/v3/projects/8c438130990f47b6b8ccd250ff03b17b/users/747ebe1303a44721be92408ced414dbb/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:02.260319 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72d62d-c5e6-4811-a54e-e8ce07eb25bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c438130990f47b6b8ccd250ff03b17b, user_id=747ebe1303a44721be92408ced414dbb, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:02.263647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72d62d-c5e6-4811-a54e-e8ce07eb25bf admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:02.288342 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 860/1725] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:02 2022] PUT /identity/v3/projects/8c438130990f47b6b8ccd250ff03b17b/users/747ebe1303a44721be92408ced414dbb/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:13:02.298851 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f53537-7960-4c1c-b05d-9a3e4fde0353 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:02.299180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f53537-7960-4c1c-b05d-9a3e4fde0353 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:02.299457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f53537-7960-4c1c-b05d-9a3e4fde0353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:02.335678 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 866/1726] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:02.341746 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a1f53537-7960-4c1c-b05d-9a3e4fde0353 None None] MFA Rules not processed for user `747ebe1303a44721be92408ced414dbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:02.343263 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a1f53537-7960-4c1c-b05d-9a3e4fde0353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:02.345683 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a356bb-d9cb-45d1-9c05-f6c12752b57f None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:02.347111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-44a356bb-d9cb-45d1-9c05-f6c12752b57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:02.417899 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a356bb-d9cb-45d1-9c05-f6c12752b57f None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:02.424120 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 861/1727] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:13:02.437355 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f53537-7960-4c1c-b05d-9a3e4fde0353 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:02.438799 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a1f53537-7960-4c1c-b05d-9a3e4fde0353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:02.482654 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce146c-9690-49f0-972b-406267cf187f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:02.483589 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce146c-9690-49f0-972b-406267cf187f service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:02.484280 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce146c-9690-49f0-972b-406267cf187f service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:02.484280 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce146c-9690-49f0-972b-406267cf187f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:02.502392 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:02.509293 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f53537-7960-4c1c-b05d-9a3e4fde0353 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:02.515153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68748b-594f-473e-8276-3224cf6b83b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:02.516060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68748b-594f-473e-8276-3224cf6b83b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:02.516360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68748b-594f-473e-8276-3224cf6b83b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:02.516687 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68748b-594f-473e-8276-3224cf6b83b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:02.526711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68748b-594f-473e-8276-3224cf6b83b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:02.528524 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68748b-594f-473e-8276-3224cf6b83b2 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:02.538270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 862/1728] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:13:02.538457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:02.540142 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68748b-594f-473e-8276-3224cf6b83b2 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:02.541364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7e68748b-594f-473e-8276-3224cf6b83b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:02.559206 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce146c-9690-49f0-972b-406267cf187f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:02.561838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce146c-9690-49f0-972b-406267cf187f service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:02.591038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 867/1729] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:02.631960 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68748b-594f-473e-8276-3224cf6b83b2 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:02.637205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346f6f0e-11bd-4d5c-b2c0-f72b81cef95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:02.638131 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f6f0e-11bd-4d5c-b2c0-f72b81cef95f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:02.638380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f6f0e-11bd-4d5c-b2c0-f72b81cef95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:02.638845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346f6f0e-11bd-4d5c-b2c0-f72b81cef95f admin admin] PATH_INFO: `/v3/projects/246520cd6cae4ce099d62e7fc85bf15d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:02.640166 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f6f0e-11bd-4d5c-b2c0-f72b81cef95f admin admin] RBAC: Authorizing `identity:delete_project(project_id=246520cd6cae4ce099d62e7fc85bf15d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:02.642219 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346f6f0e-11bd-4d5c-b2c0-f72b81cef95f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:02.697038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-346f6f0e-11bd-4d5c-b2c0-f72b81cef95f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '246520cd6cae4ce099d62e7fc85bf15d', 'request_id': 'req-346f6f0e-11bd-4d5c-b2c0-f72b81cef95f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:02.703908 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-346f6f0e-11bd-4d5c-b2c0-f72b81cef95f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '246520cd6cae4ce099d62e7fc85bf15d', 'request_id': 'req-346f6f0e-11bd-4d5c-b2c0-f72b81cef95f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:02.712715 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-346f6f0e-11bd-4d5c-b2c0-f72b81cef95f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '246520cd6cae4ce099d62e7fc85bf15d', 'request_id': 'req-346f6f0e-11bd-4d5c-b2c0-f72b81cef95f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:02.728782 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-346f6f0e-11bd-4d5c-b2c0-f72b81cef95f admin admin] The token cache is being invalidate because project 246520cd6cae4ce099d62e7fc85bf15d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:02.729110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-346f6f0e-11bd-4d5c-b2c0-f72b81cef95f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:02.729980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-346f6f0e-11bd-4d5c-b2c0-f72b81cef95f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:02.733308 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 863/1730] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:02 2022] DELETE /identity/v3/projects/246520cd6cae4ce099d62e7fc85bf15d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:02.738114 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd74b67-c726-4515-ac52-c21276655d08 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:02.738538 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd74b67-c726-4515-ac52-c21276655d08 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:02.738819 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd74b67-c726-4515-ac52-c21276655d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:02.762675 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2cd74b67-c726-4515-ac52-c21276655d08 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:02.764044 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2cd74b67-c726-4515-ac52-c21276655d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:02.846949 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 868/1731] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:02.856320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fe1e4-e46d-4e3f-86a3-add4909fa61c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:02.857785 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fe1e4-e46d-4e3f-86a3-add4909fa61c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:02.858413 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fe1e4-e46d-4e3f-86a3-add4909fa61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:02.890412 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d61fe1e4-e46d-4e3f-86a3-add4909fa61c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:02.892985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d61fe1e4-e46d-4e3f-86a3-add4909fa61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:02.911130 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 864/1732] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:02.916004 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd74b67-c726-4515-ac52-c21276655d08 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:02.930814 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd74b67-c726-4515-ac52-c21276655d08 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:02.947864 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95834e96-0163-454e-a102-a4cd2e21653c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:02.948516 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95834e96-0163-454e-a102-a4cd2e21653c admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:02.948849 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95834e96-0163-454e-a102-a4cd2e21653c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:02.949125 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95834e96-0163-454e-a102-a4cd2e21653c admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:02.949918 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95834e96-0163-454e-a102-a4cd2e21653c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:02.950223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95834e96-0163-454e-a102-a4cd2e21653c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:02.952173 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95834e96-0163-454e-a102-a4cd2e21653c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:02.981053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 869/1733] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:13:02.982628 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61fe1e4-e46d-4e3f-86a3-add4909fa61c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:02.993747 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61fe1e4-e46d-4e3f-86a3-add4909fa61c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:03.000168 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511047c7-b31f-443b-9f93-ecc3f3626177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:03.001133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511047c7-b31f-443b-9f93-ecc3f3626177 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:03.001599 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511047c7-b31f-443b-9f93-ecc3f3626177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:03.002053 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511047c7-b31f-443b-9f93-ecc3f3626177 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:03.002962 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511047c7-b31f-443b-9f93-ecc3f3626177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:03.005886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511047c7-b31f-443b-9f93-ecc3f3626177 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:03.049471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 865/1734] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:02 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:13:03.051972 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95834e96-0163-454e-a102-a4cd2e21653c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:03.063554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95834e96-0163-454e-a102-a4cd2e21653c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:03.153036 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab20296-93f6-420e-a159-59b04e79a214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:03.154189 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab20296-93f6-420e-a159-59b04e79a214 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:03.154551 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab20296-93f6-420e-a159-59b04e79a214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:03.154933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab20296-93f6-420e-a159-59b04e79a214 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:03.155799 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab20296-93f6-420e-a159-59b04e79a214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:03.158300 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab20296-93f6-420e-a159-59b04e79a214 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:03.189182 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 870/1735] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:03 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:03.193393 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511047c7-b31f-443b-9f93-ecc3f3626177 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:03.203877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511047c7-b31f-443b-9f93-ecc3f3626177 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:03.208441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06cbad0-61ed-47ca-9e2b-deab15f1958b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:03.209035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cbad0-61ed-47ca-9e2b-deab15f1958b admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:03.209845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cbad0-61ed-47ca-9e2b-deab15f1958b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:03.210117 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cbad0-61ed-47ca-9e2b-deab15f1958b admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:03.210840 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06cbad0-61ed-47ca-9e2b-deab15f1958b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:03.212525 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06cbad0-61ed-47ca-9e2b-deab15f1958b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:03.222814 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 866/1736] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:03.227525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab20296-93f6-420e-a159-59b04e79a214 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:03.237603 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab20296-93f6-420e-a159-59b04e79a214 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:03.242435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f3848-9570-496e-ab7d-8836dc0fbbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:03.243062 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f3848-9570-496e-ab7d-8836dc0fbbbb admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:03.243352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f3848-9570-496e-ab7d-8836dc0fbbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:03.243628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f3848-9570-496e-ab7d-8836dc0fbbbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:03.244267 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f3848-9570-496e-ab7d-8836dc0fbbbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:03.245866 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f3848-9570-496e-ab7d-8836dc0fbbbb admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:03.256297 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 871/1737] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:03.260642 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06cbad0-61ed-47ca-9e2b-deab15f1958b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:03.271280 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06cbad0-61ed-47ca-9e2b-deab15f1958b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:03.276485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31d34e-c1a7-4c6c-860d-84c63c05d754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:03.277143 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31d34e-c1a7-4c6c-860d-84c63c05d754 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:03.277686 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31d34e-c1a7-4c6c-860d-84c63c05d754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:03.277960 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31d34e-c1a7-4c6c-860d-84c63c05d754 admin admin] PATH_INFO: `/v3/projects/9895a9a9121d4bf197f6b1db0953f06a/users/118e9d43a40d43738fd567f25c46a0fe/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:03.298128 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31d34e-c1a7-4c6c-860d-84c63c05d754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9895a9a9121d4bf197f6b1db0953f06a, user_id=118e9d43a40d43738fd567f25c46a0fe, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:03.300841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31d34e-c1a7-4c6c-860d-84c63c05d754 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:03.314952 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 867/1738] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:03 2022] PUT /identity/v3/projects/9895a9a9121d4bf197f6b1db0953f06a/users/118e9d43a40d43738fd567f25c46a0fe/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:13:03.324945 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d655b-9d82-4ff4-9ffd-3414970510ee None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:03.325231 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d655b-9d82-4ff4-9ffd-3414970510ee None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:03.325534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d655b-9d82-4ff4-9ffd-3414970510ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:03.354397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-518d655b-9d82-4ff4-9ffd-3414970510ee None None] MFA Rules not processed for user `118e9d43a40d43738fd567f25c46a0fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:03.355654 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-518d655b-9d82-4ff4-9ffd-3414970510ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:03.396373 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:03.425595 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 872/1739] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:03.437325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31d34e-c1a7-4c6c-860d-84c63c05d754 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:03.439097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bc31d34e-c1a7-4c6c-860d-84c63c05d754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:03.461566 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:03.500452 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31d34e-c1a7-4c6c-860d-84c63c05d754 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:03.504006 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95dcfa-d30b-4b31-aa24-7ce0659f2987 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:03.504615 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95dcfa-d30b-4b31-aa24-7ce0659f2987 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:03.504897 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95dcfa-d30b-4b31-aa24-7ce0659f2987 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:03.505173 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95dcfa-d30b-4b31-aa24-7ce0659f2987 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:03.513969 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95dcfa-d30b-4b31-aa24-7ce0659f2987 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:03.515479 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95dcfa-d30b-4b31-aa24-7ce0659f2987 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:03.524927 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 868/1740] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:04.056878 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55b43a-4028-4210-8678-0b6bde4a590a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:04.057329 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55b43a-4028-4210-8678-0b6bde4a590a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:04.058232 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55b43a-4028-4210-8678-0b6bde4a590a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:04.086482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2a55b43a-4028-4210-8678-0b6bde4a590a None None] MFA Rules not processed for user `c58fbb2d79954020aa1328ef6bf925e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:04.087743 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2a55b43a-4028-4210-8678-0b6bde4a590a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:04.209370 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 873/1741] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:05.450630 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95dcfa-d30b-4b31-aa24-7ce0659f2987 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:05.452311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0b95dcfa-d30b-4b31-aa24-7ce0659f2987 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:05.525074 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95dcfa-d30b-4b31-aa24-7ce0659f2987 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:05.529876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339531c4-e818-45ad-866d-f411a732e2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:05.530519 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339531c4-e818-45ad-866d-f411a732e2ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:05.530799 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339531c4-e818-45ad-866d-f411a732e2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:05.531112 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339531c4-e818-45ad-866d-f411a732e2ac admin admin] PATH_INFO: `/v3/users/747ebe1303a44721be92408ced414dbb` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:05.532230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339531c4-e818-45ad-866d-f411a732e2ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=747ebe1303a44721be92408ced414dbb)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:05.534154 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339531c4-e818-45ad-866d-f411a732e2ac admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:05.588594 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-339531c4-e818-45ad-866d-f411a732e2ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '747ebe1303a44721be92408ced414dbb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:05.596129 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-339531c4-e818-45ad-866d-f411a732e2ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '747ebe1303a44721be92408ced414dbb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:05.597093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-339531c4-e818-45ad-866d-f411a732e2ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '747ebe1303a44721be92408ced414dbb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:05.606522 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-339531c4-e818-45ad-866d-f411a732e2ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '747ebe1303a44721be92408ced414dbb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:05.626051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-339531c4-e818-45ad-866d-f411a732e2ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '747ebe1303a44721be92408ced414dbb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:05.635085 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 869/1742] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:05 2022] DELETE /identity/v3/users/747ebe1303a44721be92408ced414dbb => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:05.639509 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59888e-7f37-4189-a5f4-354ffe1abfd1 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:05.639952 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59888e-7f37-4189-a5f4-354ffe1abfd1 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:05.640075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59888e-7f37-4189-a5f4-354ffe1abfd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:05.659620 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8d59888e-7f37-4189-a5f4-354ffe1abfd1 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:05.660846 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8d59888e-7f37-4189-a5f4-354ffe1abfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:05.733931 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 874/1743] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:05.745663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339531c4-e818-45ad-866d-f411a732e2ac admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:05.747566 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-339531c4-e818-45ad-866d-f411a732e2ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:05.818563 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:05.837231 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339531c4-e818-45ad-866d-f411a732e2ac admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:05.847708 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9ce4d-37c8-4dc3-b6fa-d2b0e00954d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:05.848348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9ce4d-37c8-4dc3-b6fa-d2b0e00954d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:05.848627 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9ce4d-37c8-4dc3-b6fa-d2b0e00954d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:05.848970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9ce4d-37c8-4dc3-b6fa-d2b0e00954d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:05.860356 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9ce4d-37c8-4dc3-b6fa-d2b0e00954d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:05.861813 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9ce4d-37c8-4dc3-b6fa-d2b0e00954d7 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:05.874716 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 870/1744] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:06.183853 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59888e-7f37-4189-a5f4-354ffe1abfd1 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:06.185251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8d59888e-7f37-4189-a5f4-354ffe1abfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:06.198538 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59888e-7f37-4189-a5f4-354ffe1abfd1 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:06.203284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae249c-ae4b-42e3-a770-a2921f4b1e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:06.204049 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae249c-ae4b-42e3-a770-a2921f4b1e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:06.204436 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae249c-ae4b-42e3-a770-a2921f4b1e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:06.204784 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae249c-ae4b-42e3-a770-a2921f4b1e1f admin admin] PATH_INFO: `/v3/projects/8c438130990f47b6b8ccd250ff03b17b` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:06.205958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae249c-ae4b-42e3-a770-a2921f4b1e1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c438130990f47b6b8ccd250ff03b17b)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:06.208081 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae249c-ae4b-42e3-a770-a2921f4b1e1f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:06.262851 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-97ae249c-ae4b-42e3-a770-a2921f4b1e1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c438130990f47b6b8ccd250ff03b17b', 'request_id': 'req-97ae249c-ae4b-42e3-a770-a2921f4b1e1f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:06.268491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-97ae249c-ae4b-42e3-a770-a2921f4b1e1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c438130990f47b6b8ccd250ff03b17b', 'request_id': 'req-97ae249c-ae4b-42e3-a770-a2921f4b1e1f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:06.275156 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-97ae249c-ae4b-42e3-a770-a2921f4b1e1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c438130990f47b6b8ccd250ff03b17b', 'request_id': 'req-97ae249c-ae4b-42e3-a770-a2921f4b1e1f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:06.288264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-97ae249c-ae4b-42e3-a770-a2921f4b1e1f admin admin] The token cache is being invalidate because project 8c438130990f47b6b8ccd250ff03b17b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:06.288583 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-97ae249c-ae4b-42e3-a770-a2921f4b1e1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:06.289224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-97ae249c-ae4b-42e3-a770-a2921f4b1e1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:06.290843 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 875/1745] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:06 2022] DELETE /identity/v3/projects/8c438130990f47b6b8ccd250ff03b17b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:06.295953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990990c-5856-4981-b891-49aa4b63430b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:06.296188 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990990c-5856-4981-b891-49aa4b63430b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:06.296468 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990990c-5856-4981-b891-49aa4b63430b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:06.316157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2990990c-5856-4981-b891-49aa4b63430b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:06.317355 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2990990c-5856-4981-b891-49aa4b63430b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:06.375313 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 871/1746] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:06.384732 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350cf1f-56c8-43c2-becd-8d448b8a32b0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:06.385023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350cf1f-56c8-43c2-becd-8d448b8a32b0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:06.385296 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350cf1f-56c8-43c2-becd-8d448b8a32b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:06.400207 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:06.406013 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f350cf1f-56c8-43c2-becd-8d448b8a32b0 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:06.407978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f350cf1f-56c8-43c2-becd-8d448b8a32b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:06.416456 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 876/1747] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:06.420780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990990c-5856-4981-b891-49aa4b63430b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:06.433503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990990c-5856-4981-b891-49aa4b63430b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:06.438453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f5152-3bf9-4b28-a4a1-155a3a7993af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:06.439232 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f5152-3bf9-4b28-a4a1-155a3a7993af admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:06.439643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f5152-3bf9-4b28-a4a1-155a3a7993af admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:06.439962 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f5152-3bf9-4b28-a4a1-155a3a7993af admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:06.440755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f5152-3bf9-4b28-a4a1-155a3a7993af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:06.441130 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f5152-3bf9-4b28-a4a1-155a3a7993af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:06.442804 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f5152-3bf9-4b28-a4a1-155a3a7993af admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:06.455757 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 872/1748] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:13:06.519671 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350cf1f-56c8-43c2-becd-8d448b8a32b0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:06.530998 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350cf1f-56c8-43c2-becd-8d448b8a32b0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:06.535970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2958ac-d423-4d7c-b886-b34de48d7acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:06.536715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2958ac-d423-4d7c-b886-b34de48d7acb admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:06.537066 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2958ac-d423-4d7c-b886-b34de48d7acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:06.538297 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2958ac-d423-4d7c-b886-b34de48d7acb admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:06.543351 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2958ac-d423-4d7c-b886-b34de48d7acb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:06.546550 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2958ac-d423-4d7c-b886-b34de48d7acb admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:06.581332 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f5152-3bf9-4b28-a4a1-155a3a7993af admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:06.588013 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 877/1749] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:06 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:13:06.592484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f5152-3bf9-4b28-a4a1-155a3a7993af admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:06.682052 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2958ac-d423-4d7c-b886-b34de48d7acb admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:06.684294 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2d3f3-8ba9-4d17-bf96-b6d68408682c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:06.684439 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2d2958ac-d423-4d7c-b886-b34de48d7acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:06.685048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2d3f3-8ba9-4d17-bf96-b6d68408682c admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:06.685336 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2d3f3-8ba9-4d17-bf96-b6d68408682c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:06.685687 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2d3f3-8ba9-4d17-bf96-b6d68408682c admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:06.686484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2d3f3-8ba9-4d17-bf96-b6d68408682c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:06.689573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e2d3f3-8ba9-4d17-bf96-b6d68408682c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:06.707887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2958ac-d423-4d7c-b886-b34de48d7acb admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:06.717675 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 873/1750] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:06 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:13:06.718625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf96ca6-9b8b-42c0-ae7a-29e4e0fe261b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:06.719502 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf96ca6-9b8b-42c0-ae7a-29e4e0fe261b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:06.719782 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf96ca6-9b8b-42c0-ae7a-29e4e0fe261b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:06.720445 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf96ca6-9b8b-42c0-ae7a-29e4e0fe261b admin admin] PATH_INFO: `/v3/users/118e9d43a40d43738fd567f25c46a0fe` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:06.721814 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf96ca6-9b8b-42c0-ae7a-29e4e0fe261b admin admin] RBAC: Authorizing `identity:delete_user(user_id=118e9d43a40d43738fd567f25c46a0fe)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:06.721936 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2d3f3-8ba9-4d17-bf96-b6d68408682c admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:06.724574 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf96ca6-9b8b-42c0-ae7a-29e4e0fe261b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:06.733488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2d3f3-8ba9-4d17-bf96-b6d68408682c admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:06.739094 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d0032-ca66-4f38-a0f9-4ca6120ffdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:06.739709 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d0032-ca66-4f38-a0f9-4ca6120ffdd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:06.739986 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d0032-ca66-4f38-a0f9-4ca6120ffdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:06.740253 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d0032-ca66-4f38-a0f9-4ca6120ffdd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:06.741311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d0032-ca66-4f38-a0f9-4ca6120ffdd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:06.742935 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1d0032-ca66-4f38-a0f9-4ca6120ffdd1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:06.766941 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 874/1751] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:06.771348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d0032-ca66-4f38-a0f9-4ca6120ffdd1 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:06.794555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-aaf96ca6-9b8b-42c0-ae7a-29e4e0fe261b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '118e9d43a40d43738fd567f25c46a0fe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:06.799556 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d0032-ca66-4f38-a0f9-4ca6120ffdd1 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:06.804677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-aaf96ca6-9b8b-42c0-ae7a-29e4e0fe261b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '118e9d43a40d43738fd567f25c46a0fe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:06.805822 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-aaf96ca6-9b8b-42c0-ae7a-29e4e0fe261b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '118e9d43a40d43738fd567f25c46a0fe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:06.806922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a94fb1-7321-4748-afaf-1cd36ca948a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:06.807671 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a94fb1-7321-4748-afaf-1cd36ca948a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:06.807986 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a94fb1-7321-4748-afaf-1cd36ca948a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:06.808275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a94fb1-7321-4748-afaf-1cd36ca948a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:06.808979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a94fb1-7321-4748-afaf-1cd36ca948a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:06.810881 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a94fb1-7321-4748-afaf-1cd36ca948a3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:06.817485 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-aaf96ca6-9b8b-42c0-ae7a-29e4e0fe261b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '118e9d43a40d43738fd567f25c46a0fe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:06.826604 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 875/1752] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:06.830299 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a94fb1-7321-4748-afaf-1cd36ca948a3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:06.837766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e0a94fb1-7321-4748-afaf-1cd36ca948a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:06.854195 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-aaf96ca6-9b8b-42c0-ae7a-29e4e0fe261b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '118e9d43a40d43738fd567f25c46a0fe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:06.867245 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a94fb1-7321-4748-afaf-1cd36ca948a3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:06.868810 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 878/1753] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:06 2022] DELETE /identity/v3/users/118e9d43a40d43738fd567f25c46a0fe => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:13:06.874463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e165e4-17ab-4b99-89bf-b96b926829e9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:06.874763 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e165e4-17ab-4b99-89bf-b96b926829e9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:06.875080 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e165e4-17ab-4b99-89bf-b96b926829e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:06.897648 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-55e165e4-17ab-4b99-89bf-b96b926829e9 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:06.899840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-55e165e4-17ab-4b99-89bf-b96b926829e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:06.904114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:06.934645 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1eef8-14e9-400d-8a21-1245f04a2718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:06.935360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1eef8-14e9-400d-8a21-1245f04a2718 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:06.935644 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1eef8-14e9-400d-8a21-1245f04a2718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:06.935922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1eef8-14e9-400d-8a21-1245f04a2718 admin admin] PATH_INFO: `/v3/projects/0d285ecc0c5343b1b4001594013cd447/users/7cc092c3cc0b4fbc9b420b8c7916132a/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:06.955118 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1eef8-14e9-400d-8a21-1245f04a2718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d285ecc0c5343b1b4001594013cd447, user_id=7cc092c3cc0b4fbc9b420b8c7916132a, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:06.959168 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1eef8-14e9-400d-8a21-1245f04a2718 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:06.978997 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 876/1754] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:06 2022] PUT /identity/v3/projects/0d285ecc0c5343b1b4001594013cd447/users/7cc092c3cc0b4fbc9b420b8c7916132a/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:06.981017 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec100314-7f22-417d-9804-36595a79d664 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:06.981359 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec100314-7f22-417d-9804-36595a79d664 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:06.981652 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec100314-7f22-417d-9804-36595a79d664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:06.982554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 879/1755] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:13:06.993771 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e165e4-17ab-4b99-89bf-b96b926829e9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:06.996792 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-55e165e4-17ab-4b99-89bf-b96b926829e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:07.007511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ec100314-7f22-417d-9804-36595a79d664 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:07.009000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ec100314-7f22-417d-9804-36595a79d664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:07.083932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e165e4-17ab-4b99-89bf-b96b926829e9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:07.091491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:07.094078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 877/1756] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:07.107944 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec100314-7f22-417d-9804-36595a79d664 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:07.170938 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcaec2b-ca98-4c6e-94e8-246122a92c80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:07.171805 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaec2b-ca98-4c6e-94e8-246122a92c80 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:07.172235 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaec2b-ca98-4c6e-94e8-246122a92c80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:07.172680 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaec2b-ca98-4c6e-94e8-246122a92c80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:07.183589 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcaec2b-ca98-4c6e-94e8-246122a92c80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:07.185082 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcaec2b-ca98-4c6e-94e8-246122a92c80 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:07.196224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 880/1757] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:07.199193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec100314-7f22-417d-9804-36595a79d664 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:07.204019 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2022838-ef0f-4343-be29-ae94963fb63e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:07.204019 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2022838-ef0f-4343-be29-ae94963fb63e service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:07.205178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2022838-ef0f-4343-be29-ae94963fb63e service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:07.205566 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2022838-ef0f-4343-be29-ae94963fb63e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:07.223040 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2022838-ef0f-4343-be29-ae94963fb63e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:07.228353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2022838-ef0f-4343-be29-ae94963fb63e service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:07.246829 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 878/1758] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:07.659678 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcaec2b-ca98-4c6e-94e8-246122a92c80 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:07.661345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-abcaec2b-ca98-4c6e-94e8-246122a92c80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:07.675824 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcaec2b-ca98-4c6e-94e8-246122a92c80 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:07.681667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef50c4-f81f-4ac8-9fe6-1fd39c04d0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:07.682379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef50c4-f81f-4ac8-9fe6-1fd39c04d0d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:07.683003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef50c4-f81f-4ac8-9fe6-1fd39c04d0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:07.683003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef50c4-f81f-4ac8-9fe6-1fd39c04d0d1 admin admin] PATH_INFO: `/v3/projects/9895a9a9121d4bf197f6b1db0953f06a` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:07.684264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef50c4-f81f-4ac8-9fe6-1fd39c04d0d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9895a9a9121d4bf197f6b1db0953f06a)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:07.686287 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef50c4-f81f-4ac8-9fe6-1fd39c04d0d1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:07.745242 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c8ef50c4-f81f-4ac8-9fe6-1fd39c04d0d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9895a9a9121d4bf197f6b1db0953f06a', 'request_id': 'req-c8ef50c4-f81f-4ac8-9fe6-1fd39c04d0d1'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:07.753574 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c8ef50c4-f81f-4ac8-9fe6-1fd39c04d0d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9895a9a9121d4bf197f6b1db0953f06a', 'request_id': 'req-c8ef50c4-f81f-4ac8-9fe6-1fd39c04d0d1'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:07.761732 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c8ef50c4-f81f-4ac8-9fe6-1fd39c04d0d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9895a9a9121d4bf197f6b1db0953f06a', 'request_id': 'req-c8ef50c4-f81f-4ac8-9fe6-1fd39c04d0d1'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:07.777346 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c8ef50c4-f81f-4ac8-9fe6-1fd39c04d0d1 admin admin] The token cache is being invalidate because project 9895a9a9121d4bf197f6b1db0953f06a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:07.778243 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c8ef50c4-f81f-4ac8-9fe6-1fd39c04d0d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:07.779922 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c8ef50c4-f81f-4ac8-9fe6-1fd39c04d0d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:07.782026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 881/1759] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:07 2022] DELETE /identity/v3/projects/9895a9a9121d4bf197f6b1db0953f06a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:07.788421 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54902a4b-1bee-4ae7-867f-2d72112ffeda None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:07.788897 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54902a4b-1bee-4ae7-867f-2d72112ffeda None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:07.789137 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54902a4b-1bee-4ae7-867f-2d72112ffeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:07.825538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-54902a4b-1bee-4ae7-867f-2d72112ffeda None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:07.826980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-54902a4b-1bee-4ae7-867f-2d72112ffeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:07.928873 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 879/1760] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:13:07.939228 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f555be-90dc-4361-94ab-7922a1a26685 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:07.939795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f555be-90dc-4361-94ab-7922a1a26685 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:07.940078 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f555be-90dc-4361-94ab-7922a1a26685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:07.968374 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-97f555be-90dc-4361-94ab-7922a1a26685 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:07.975721 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-97f555be-90dc-4361-94ab-7922a1a26685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:07.986663 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 882/1761] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:07.990955 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54902a4b-1bee-4ae7-867f-2d72112ffeda None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:07.997506 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:08.002365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54902a4b-1bee-4ae7-867f-2d72112ffeda None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:08.011317 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b58f4-ff9d-49aa-bb27-235490cb973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:08.012020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b58f4-ff9d-49aa-bb27-235490cb973e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:08.012377 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b58f4-ff9d-49aa-bb27-235490cb973e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:08.012744 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b58f4-ff9d-49aa-bb27-235490cb973e admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:08.013861 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b58f4-ff9d-49aa-bb27-235490cb973e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:08.013861 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b58f4-ff9d-49aa-bb27-235490cb973e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:08.015610 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b58f4-ff9d-49aa-bb27-235490cb973e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:08.032802 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 880/1762] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:13:08.034033 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f555be-90dc-4361-94ab-7922a1a26685 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:08.043696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f555be-90dc-4361-94ab-7922a1a26685 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:08.049944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585626f-3740-4c23-ae37-26fe4a92f324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:08.049944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585626f-3740-4c23-ae37-26fe4a92f324 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:08.049944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585626f-3740-4c23-ae37-26fe4a92f324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:08.050243 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585626f-3740-4c23-ae37-26fe4a92f324 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:08.050784 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585626f-3740-4c23-ae37-26fe4a92f324 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:08.052800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585626f-3740-4c23-ae37-26fe4a92f324 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:08.086924 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 883/1763] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:08 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:13:08.090977 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b58f4-ff9d-49aa-bb27-235490cb973e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:08.103608 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b58f4-ff9d-49aa-bb27-235490cb973e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:08.178689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3b357-ccd2-470b-b0de-9b47ba212c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:08.179514 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3b357-ccd2-470b-b0de-9b47ba212c56 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:08.179974 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3b357-ccd2-470b-b0de-9b47ba212c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:08.180795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3b357-ccd2-470b-b0de-9b47ba212c56 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:08.181377 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3b357-ccd2-470b-b0de-9b47ba212c56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:08.184622 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3b357-ccd2-470b-b0de-9b47ba212c56 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:08.221548 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 881/1764] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:08 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:08.227260 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585626f-3740-4c23-ae37-26fe4a92f324 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:08.250786 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585626f-3740-4c23-ae37-26fe4a92f324 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:08.258770 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd79b7-90a5-4061-b0ae-fcdaad6d3b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:08.259805 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd79b7-90a5-4061-b0ae-fcdaad6d3b6e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:08.260190 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd79b7-90a5-4061-b0ae-fcdaad6d3b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:08.260563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd79b7-90a5-4061-b0ae-fcdaad6d3b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:08.262293 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd79b7-90a5-4061-b0ae-fcdaad6d3b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:08.265480 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd79b7-90a5-4061-b0ae-fcdaad6d3b6e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:08.277822 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 884/1765] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:13:08.282438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3b357-ccd2-470b-b0de-9b47ba212c56 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:08.296065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3b357-ccd2-470b-b0de-9b47ba212c56 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:08.301365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d091302-7d4c-4cd8-aa09-e95a5ec21c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:08.301987 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d091302-7d4c-4cd8-aa09-e95a5ec21c99 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:08.302268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d091302-7d4c-4cd8-aa09-e95a5ec21c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:08.303190 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d091302-7d4c-4cd8-aa09-e95a5ec21c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:08.303602 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d091302-7d4c-4cd8-aa09-e95a5ec21c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:08.305570 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d091302-7d4c-4cd8-aa09-e95a5ec21c99 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:08.321663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 882/1766] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:08.322988 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd79b7-90a5-4061-b0ae-fcdaad6d3b6e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:08.333431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd79b7-90a5-4061-b0ae-fcdaad6d3b6e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:08.339596 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cca4ad-fba5-4813-a197-8320b518fc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:08.340459 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cca4ad-fba5-4813-a197-8320b518fc57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:08.340750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cca4ad-fba5-4813-a197-8320b518fc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:08.341039 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cca4ad-fba5-4813-a197-8320b518fc57 admin admin] PATH_INFO: `/v3/projects/fc4fa086ae1647e48281586eba4c06e2/users/eec2a2cdcafd40cd8b8e6f80c3c54191/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:08.362136 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cca4ad-fba5-4813-a197-8320b518fc57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc4fa086ae1647e48281586eba4c06e2, user_id=eec2a2cdcafd40cd8b8e6f80c3c54191, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:08.366592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cca4ad-fba5-4813-a197-8320b518fc57 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:08.389317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 885/1767] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:08 2022] PUT /identity/v3/projects/fc4fa086ae1647e48281586eba4c06e2/users/eec2a2cdcafd40cd8b8e6f80c3c54191/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:08.400691 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962aa487-96f9-4687-a724-2f0ab83048af None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:08.401270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962aa487-96f9-4687-a724-2f0ab83048af None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:08.401554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962aa487-96f9-4687-a724-2f0ab83048af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:08.434211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e91d02-776a-4940-b33f-57df14cf7353 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:08.434529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e91d02-776a-4940-b33f-57df14cf7353 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:08.435011 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e91d02-776a-4940-b33f-57df14cf7353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:08.469012 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-962aa487-96f9-4687-a724-2f0ab83048af None None] MFA Rules not processed for user `eec2a2cdcafd40cd8b8e6f80c3c54191`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:08.470645 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-962aa487-96f9-4687-a724-2f0ab83048af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:08.518909 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d6e91d02-776a-4940-b33f-57df14cf7353 None None] MFA Rules not processed for user `7cc092c3cc0b4fbc9b420b8c7916132a`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:08.520055 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d6e91d02-776a-4940-b33f-57df14cf7353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:08.546697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:08.578647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 883/1768] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:08.584951 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962aa487-96f9-4687-a724-2f0ab83048af None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:08.611959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 886/1769] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:13:08.627140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e91d02-776a-4940-b33f-57df14cf7353 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:08.629549 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d6e91d02-776a-4940-b33f-57df14cf7353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:08.679821 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962aa487-96f9-4687-a724-2f0ab83048af None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:08.686290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82d6e5-af47-4fce-b72d-bb6617e95b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:08.687824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82d6e5-af47-4fce-b72d-bb6617e95b0f admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:08.688217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82d6e5-af47-4fce-b72d-bb6617e95b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:08.688619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82d6e5-af47-4fce-b72d-bb6617e95b0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:08.689300 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82d6e5-af47-4fce-b72d-bb6617e95b0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:08.692217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82d6e5-af47-4fce-b72d-bb6617e95b0f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:08.699548 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:08.745127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 884/1770] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:08 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:08.749186 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82d6e5-af47-4fce-b72d-bb6617e95b0f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:08.761057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e91d02-776a-4940-b33f-57df14cf7353 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:08.821217 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42408c73-1c4d-4818-836c-f9a2d45922a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:08.822159 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42408c73-1c4d-4818-836c-f9a2d45922a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:08.822440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42408c73-1c4d-4818-836c-f9a2d45922a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:08.822717 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42408c73-1c4d-4818-836c-f9a2d45922a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:08.876459 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82d6e5-af47-4fce-b72d-bb6617e95b0f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:08.886034 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9ffb1-97cf-44ec-82ca-c379d12701c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:08.887866 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9ffb1-97cf-44ec-82ca-c379d12701c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:08.887866 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9ffb1-97cf-44ec-82ca-c379d12701c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:08.887866 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9ffb1-97cf-44ec-82ca-c379d12701c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:08.888327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c9ffb1-97cf-44ec-82ca-c379d12701c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:08.892219 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c9ffb1-97cf-44ec-82ca-c379d12701c7 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:08.893585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42408c73-1c4d-4818-836c-f9a2d45922a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:08.895315 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42408c73-1c4d-4818-836c-f9a2d45922a3 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:08.908810 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 887/1771] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:08.941685 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 885/1772] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:08 2022] POST /identity/v3/users => generated 292 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:08.942762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42408c73-1c4d-4818-836c-f9a2d45922a3 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:08.961003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42408c73-1c4d-4818-836c-f9a2d45922a3 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:08.970641 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc1b4f-bb10-473a-ac7e-8e2189ce6dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:08.971298 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc1b4f-bb10-473a-ac7e-8e2189ce6dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:08.971582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc1b4f-bb10-473a-ac7e-8e2189ce6dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:08.971862 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc1b4f-bb10-473a-ac7e-8e2189ce6dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:08.972510 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc1b4f-bb10-473a-ac7e-8e2189ce6dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:08.974395 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc1b4f-bb10-473a-ac7e-8e2189ce6dc7 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:08.999335 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 888/1773] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:13:09.000768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9ffb1-97cf-44ec-82ca-c379d12701c7 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:09.016704 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9ffb1-97cf-44ec-82ca-c379d12701c7 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:09.022805 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4bb61-89bb-4734-a240-6f5035a34e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:09.023453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4bb61-89bb-4734-a240-6f5035a34e8b admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:09.023736 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4bb61-89bb-4734-a240-6f5035a34e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:09.024010 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4bb61-89bb-4734-a240-6f5035a34e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:09.024732 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4bb61-89bb-4734-a240-6f5035a34e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:09.027236 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4bb61-89bb-4734-a240-6f5035a34e8b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:09.043928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 886/1774] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:09.044926 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc1b4f-bb10-473a-ac7e-8e2189ce6dc7 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:09.059241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc1b4f-bb10-473a-ac7e-8e2189ce6dc7 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:09.065935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb9341-87da-4f6c-87ca-6db88c51a1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:09.066613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb9341-87da-4f6c-87ca-6db88c51a1da admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:09.066911 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb9341-87da-4f6c-87ca-6db88c51a1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:09.067189 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb9341-87da-4f6c-87ca-6db88c51a1da admin admin] PATH_INFO: `/v3/projects/b883e063e75a4a82929da6bd4a713124/users/7194a60f95e4409f9847694a21d12721/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:09.084834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb9341-87da-4f6c-87ca-6db88c51a1da admin admin] RBAC: Authorizing `identity:create_grant(project_id=b883e063e75a4a82929da6bd4a713124, user_id=7194a60f95e4409f9847694a21d12721, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:09.087611 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb9341-87da-4f6c-87ca-6db88c51a1da admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:09.089988 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:09.107145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 889/1775] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:09 2022] PUT /identity/v3/projects/b883e063e75a4a82929da6bd4a713124/users/7194a60f95e4409f9847694a21d12721/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:13:09.118175 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9d950-7742-4eb8-bfd0-13c37052afd2 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:09.118764 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9d950-7742-4eb8-bfd0-13c37052afd2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:09.119086 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9d950-7742-4eb8-bfd0-13c37052afd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:09.169480 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a3c9d950-7742-4eb8-bfd0-13c37052afd2 None None] MFA Rules not processed for user `7194a60f95e4409f9847694a21d12721`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:09.172742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a3c9d950-7742-4eb8-bfd0-13c37052afd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:09.266936 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 887/1776] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:09.272824 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb9341-87da-4f6c-87ca-6db88c51a1da admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:09.359362 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb9341-87da-4f6c-87ca-6db88c51a1da admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:09.364333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bede963-43b6-4d99-be4a-9d913899ffd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:09.365296 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bede963-43b6-4d99-be4a-9d913899ffd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:09.366147 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bede963-43b6-4d99-be4a-9d913899ffd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:09.366508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bede963-43b6-4d99-be4a-9d913899ffd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:09.367418 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bede963-43b6-4d99-be4a-9d913899ffd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:09.369500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bede963-43b6-4d99-be4a-9d913899ffd5 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:09.400083 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 890/1777] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:09 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:09.404843 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9d950-7742-4eb8-bfd0-13c37052afd2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:09.416587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9d950-7742-4eb8-bfd0-13c37052afd2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:09.466406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:09.477921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65f562-bc16-476d-8bb9-9a939c586dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:09.478537 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65f562-bc16-476d-8bb9-9a939c586dab admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:09.478820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65f562-bc16-476d-8bb9-9a939c586dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:09.479127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65f562-bc16-476d-8bb9-9a939c586dab admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:09.479767 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65f562-bc16-476d-8bb9-9a939c586dab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:09.481741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65f562-bc16-476d-8bb9-9a939c586dab admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:09.503436 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 888/1778] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:09 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:09.508041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bede963-43b6-4d99-be4a-9d913899ffd5 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:09.518484 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bede963-43b6-4d99-be4a-9d913899ffd5 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:09.523514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f128a-6c77-4af6-9be5-2d361c5004ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:09.524128 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f128a-6c77-4af6-9be5-2d361c5004ca admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:09.524409 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f128a-6c77-4af6-9be5-2d361c5004ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:09.524687 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f128a-6c77-4af6-9be5-2d361c5004ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:09.525331 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f128a-6c77-4af6-9be5-2d361c5004ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:09.526985 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f128a-6c77-4af6-9be5-2d361c5004ca admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:09.538732 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 891/1779] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:09.543116 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65f562-bc16-476d-8bb9-9a939c586dab admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:09.554208 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65f562-bc16-476d-8bb9-9a939c586dab admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:09.559062 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1086953-ad02-4ebf-82c7-415239f54c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:09.559725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1086953-ad02-4ebf-82c7-415239f54c85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:09.560007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1086953-ad02-4ebf-82c7-415239f54c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:09.560282 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1086953-ad02-4ebf-82c7-415239f54c85 admin admin] PATH_INFO: `/v3/projects/0dc8dfcd2640459e9b9b18b3df82db38/users/223ab369468743699d84b4d0a8741bb8/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:09.575700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1086953-ad02-4ebf-82c7-415239f54c85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc8dfcd2640459e9b9b18b3df82db38, user_id=223ab369468743699d84b4d0a8741bb8, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:09.578386 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1086953-ad02-4ebf-82c7-415239f54c85 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:09.593850 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 889/1780] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:09 2022] PUT /identity/v3/projects/0dc8dfcd2640459e9b9b18b3df82db38/users/223ab369468743699d84b4d0a8741bb8/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:09.597740 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f128a-6c77-4af6-9be5-2d361c5004ca admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:09.608213 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f128a-6c77-4af6-9be5-2d361c5004ca admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:09.674370 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca1259-8601-475e-a686-89eaa4405e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:09.674932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca1259-8601-475e-a686-89eaa4405e1b admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:09.675214 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca1259-8601-475e-a686-89eaa4405e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:09.675491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca1259-8601-475e-a686-89eaa4405e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:09.676188 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca1259-8601-475e-a686-89eaa4405e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:09.677773 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca1259-8601-475e-a686-89eaa4405e1b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:09.690777 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 892/1781] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:09.695449 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1086953-ad02-4ebf-82c7-415239f54c85 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:09.721554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1086953-ad02-4ebf-82c7-415239f54c85 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:09.727469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc8678-d0f4-4ada-a074-29bc95fd69c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:09.728144 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc8678-d0f4-4ada-a074-29bc95fd69c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:09.728427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc8678-d0f4-4ada-a074-29bc95fd69c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:09.728705 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc8678-d0f4-4ada-a074-29bc95fd69c5 admin admin] PATH_INFO: `/v3/projects/0dc8dfcd2640459e9b9b18b3df82db38/users/223ab369468743699d84b4d0a8741bb8/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:09.731022 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc8678-d0f4-4ada-a074-29bc95fd69c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc8dfcd2640459e9b9b18b3df82db38, user_id=223ab369468743699d84b4d0a8741bb8, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:09.735135 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc8678-d0f4-4ada-a074-29bc95fd69c5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:09.748362 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 890/1782] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:09 2022] PUT /identity/v3/projects/0dc8dfcd2640459e9b9b18b3df82db38/users/223ab369468743699d84b4d0a8741bb8/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:09.760097 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef64ff-df46-4ef0-b122-96a60a07ee17 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:09.760371 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef64ff-df46-4ef0-b122-96a60a07ee17 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:09.760645 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef64ff-df46-4ef0-b122-96a60a07ee17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:09.794460 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-efef64ff-df46-4ef0-b122-96a60a07ee17 None None] MFA Rules not processed for user `223ab369468743699d84b4d0a8741bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:09.796544 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-efef64ff-df46-4ef0-b122-96a60a07ee17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:09.858801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:09.882562 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 893/1783] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:09.895999 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc8678-d0f4-4ada-a074-29bc95fd69c5 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:09.964157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc8678-d0f4-4ada-a074-29bc95fd69c5 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:09.968888 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93094ec4-5d55-4cdf-a3ea-ad888ed822a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:09.970460 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93094ec4-5d55-4cdf-a3ea-ad888ed822a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:09.983401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93094ec4-5d55-4cdf-a3ea-ad888ed822a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:09.983767 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93094ec4-5d55-4cdf-a3ea-ad888ed822a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:10.074479 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93094ec4-5d55-4cdf-a3ea-ad888ed822a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:10.076042 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93094ec4-5d55-4cdf-a3ea-ad888ed822a3 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:10.088654 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 891/1784] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:10.429433 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe565530-6166-4e40-b813-532fa26cb756 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:10.429776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe565530-6166-4e40-b813-532fa26cb756 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:10.429994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe565530-6166-4e40-b813-532fa26cb756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:10.481717 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-fe565530-6166-4e40-b813-532fa26cb756 None None] MFA Rules not processed for user `7cc092c3cc0b4fbc9b420b8c7916132a`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:10.495226 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fe565530-6166-4e40-b813-532fa26cb756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:10.572075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 894/1785] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:11.820012 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93094ec4-5d55-4cdf-a3ea-ad888ed822a3 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:11.821540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-93094ec4-5d55-4cdf-a3ea-ad888ed822a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:11.833370 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93094ec4-5d55-4cdf-a3ea-ad888ed822a3 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:11.837402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d05f8-d44e-463a-b008-bd353e78b5c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:11.838012 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d05f8-d44e-463a-b008-bd353e78b5c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:11.838292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d05f8-d44e-463a-b008-bd353e78b5c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:11.838701 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d05f8-d44e-463a-b008-bd353e78b5c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:11.915078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d05f8-d44e-463a-b008-bd353e78b5c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:11.916918 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d05f8-d44e-463a-b008-bd353e78b5c3 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:11.937021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:11.947943 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 892/1786] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:13:12.898116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe565530-6166-4e40-b813-532fa26cb756 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:12.909786 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe565530-6166-4e40-b813-532fa26cb756 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:12.914386 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474afd14-7879-4e64-9351-d240e5c4369e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:12.915336 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474afd14-7879-4e64-9351-d240e5c4369e service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:12.916186 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474afd14-7879-4e64-9351-d240e5c4369e service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:12.916675 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474afd14-7879-4e64-9351-d240e5c4369e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:12.929584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474afd14-7879-4e64-9351-d240e5c4369e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:12.931087 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474afd14-7879-4e64-9351-d240e5c4369e service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:12.942907 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 895/1787] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:13.156047 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37024a05-42c6-49ef-ba01-3272a7cf4440 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:13.156906 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37024a05-42c6-49ef-ba01-3272a7cf4440 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:13.157334 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37024a05-42c6-49ef-ba01-3272a7cf4440 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:13.159373 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 893/1788] 10.222.0.27 () {54 vars in 921 bytes} [Mon Oct 17 03:13:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:13:13.207835 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4e4e3-e449-4093-8e3c-d7cef1480eb7 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:13.208129 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4e4e3-e449-4093-8e3c-d7cef1480eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:13.208471 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4e4e3-e449-4093-8e3c-d7cef1480eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:13.246467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4ea4e4e3-e449-4093-8e3c-d7cef1480eb7 None None] MFA Rules not processed for user `4cc2655cc32e4d63a7000ef41bcbcfea`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:13.248088 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4ea4e4e3-e449-4093-8e3c-d7cef1480eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:13.258540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37024a05-42c6-49ef-ba01-3272a7cf4440 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:13.261523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-37024a05-42c6-49ef-ba01-3272a7cf4440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:13.345588 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 896/1789] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Oct 17 03:13:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:13:13.358721 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4e4e3-e449-4093-8e3c-d7cef1480eb7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:13.360101 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4ea4e4e3-e449-4093-8e3c-d7cef1480eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:13.367770 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37024a05-42c6-49ef-ba01-3272a7cf4440 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:13.373423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af618db-0b8e-4181-9aef-d5f3f0e76497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:13.374132 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af618db-0b8e-4181-9aef-d5f3f0e76497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:13.374482 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af618db-0b8e-4181-9aef-d5f3f0e76497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:13.374826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af618db-0b8e-4181-9aef-d5f3f0e76497 admin admin] PATH_INFO: `/v3/users/4443729a5a5341b6ad64f6fa131a5a54` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:13.375220 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4e4e3-e449-4093-8e3c-d7cef1480eb7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:13.376160 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af618db-0b8e-4181-9aef-d5f3f0e76497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4443729a5a5341b6ad64f6fa131a5a54)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:13.378142 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af618db-0b8e-4181-9aef-d5f3f0e76497 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:13.381908 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69f364-f342-400c-870f-9c6f1e9b286b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cc2655cc32e4d63a7000ef41bcbcfea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:13.382673 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69f364-f342-400c-870f-9c6f1e9b286b service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:13.383076 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69f364-f342-400c-870f-9c6f1e9b286b service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:13.383450 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69f364-f342-400c-870f-9c6f1e9b286b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:13.393395 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a69f364-f342-400c-870f-9c6f1e9b286b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:13.395247 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a69f364-f342-400c-870f-9c6f1e9b286b service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:13.408234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 897/1790] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:13:13.449438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7af618db-0b8e-4181-9aef-d5f3f0e76497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4443729a5a5341b6ad64f6fa131a5a54'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:13.456637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7af618db-0b8e-4181-9aef-d5f3f0e76497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4443729a5a5341b6ad64f6fa131a5a54'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:13.457323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7af618db-0b8e-4181-9aef-d5f3f0e76497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4443729a5a5341b6ad64f6fa131a5a54'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:13.465655 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7af618db-0b8e-4181-9aef-d5f3f0e76497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4443729a5a5341b6ad64f6fa131a5a54'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:13.478133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7af618db-0b8e-4181-9aef-d5f3f0e76497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4443729a5a5341b6ad64f6fa131a5a54'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:13.484214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 894/1791] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:13 2022] DELETE /identity/v3/users/4443729a5a5341b6ad64f6fa131a5a54 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:13.657605 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69f364-f342-400c-870f-9c6f1e9b286b service placement] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:13.670132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69f364-f342-400c-870f-9c6f1e9b286b service placement] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:13.738302 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763349b2-6972-4234-9309-6f4e19d68f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:13.739264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763349b2-6972-4234-9309-6f4e19d68f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:13.739554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763349b2-6972-4234-9309-6f4e19d68f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:13.739949 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763349b2-6972-4234-9309-6f4e19d68f0e admin admin] PATH_INFO: `/v3/projects/d03f03d21e9e42fb91606306241a3fb7` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:13.742344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763349b2-6972-4234-9309-6f4e19d68f0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d03f03d21e9e42fb91606306241a3fb7)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:13.744382 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763349b2-6972-4234-9309-6f4e19d68f0e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:13.750983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:13.811550 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-763349b2-6972-4234-9309-6f4e19d68f0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd03f03d21e9e42fb91606306241a3fb7', 'request_id': 'req-763349b2-6972-4234-9309-6f4e19d68f0e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:13.818389 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-763349b2-6972-4234-9309-6f4e19d68f0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd03f03d21e9e42fb91606306241a3fb7', 'request_id': 'req-763349b2-6972-4234-9309-6f4e19d68f0e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:13.826098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-763349b2-6972-4234-9309-6f4e19d68f0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd03f03d21e9e42fb91606306241a3fb7', 'request_id': 'req-763349b2-6972-4234-9309-6f4e19d68f0e'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:13.843214 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-763349b2-6972-4234-9309-6f4e19d68f0e admin admin] The token cache is being invalidate because project d03f03d21e9e42fb91606306241a3fb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:13.843905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-763349b2-6972-4234-9309-6f4e19d68f0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:13.845012 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-763349b2-6972-4234-9309-6f4e19d68f0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:13.847140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 898/1792] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:13 2022] DELETE /identity/v3/projects/d03f03d21e9e42fb91606306241a3fb7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:13.852606 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ee47d-33f0-44dd-a5e3-0abc971ff8d4 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:13.853094 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ee47d-33f0-44dd-a5e3-0abc971ff8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:13.853516 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ee47d-33f0-44dd-a5e3-0abc971ff8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:13.878443 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b14ee47d-33f0-44dd-a5e3-0abc971ff8d4 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:13.879624 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b14ee47d-33f0-44dd-a5e3-0abc971ff8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:13.963509 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 895/1793] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:13.971406 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f61a57-fbea-4883-abb8-628bc951bb6f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:13.971917 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f61a57-fbea-4883-abb8-628bc951bb6f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:13.972125 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f61a57-fbea-4883-abb8-628bc951bb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:13.996282 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-94f61a57-fbea-4883-abb8-628bc951bb6f None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:13.997546 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-94f61a57-fbea-4883-abb8-628bc951bb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:14.006140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 899/1794] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:14.010584 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ee47d-33f0-44dd-a5e3-0abc971ff8d4 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:14.020663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ee47d-33f0-44dd-a5e3-0abc971ff8d4 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:14.025436 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cffa2da-1810-4c6a-94b2-aeaddd642883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:14.026066 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffa2da-1810-4c6a-94b2-aeaddd642883 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:14.026335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffa2da-1810-4c6a-94b2-aeaddd642883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:14.026611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cffa2da-1810-4c6a-94b2-aeaddd642883 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:14.027340 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cffa2da-1810-4c6a-94b2-aeaddd642883 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:14.027892 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cffa2da-1810-4c6a-94b2-aeaddd642883 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:14.029523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cffa2da-1810-4c6a-94b2-aeaddd642883 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:14.042222 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 896/1795] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:13:14.047442 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f61a57-fbea-4883-abb8-628bc951bb6f None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:14.057958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f61a57-fbea-4883-abb8-628bc951bb6f None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:14.064934 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466cfe0-1397-48c5-aea3-83f15b33181f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:14.065555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466cfe0-1397-48c5-aea3-83f15b33181f admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:14.065834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466cfe0-1397-48c5-aea3-83f15b33181f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:14.066169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466cfe0-1397-48c5-aea3-83f15b33181f admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:14.066818 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3466cfe0-1397-48c5-aea3-83f15b33181f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:14.069690 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3466cfe0-1397-48c5-aea3-83f15b33181f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:14.102181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 900/1796] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:14 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:14.103811 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cffa2da-1810-4c6a-94b2-aeaddd642883 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:14.111018 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:14.115405 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cffa2da-1810-4c6a-94b2-aeaddd642883 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:14.185119 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4818d-46e8-43af-8192-32a2ffbd451b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:14.186467 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4818d-46e8-43af-8192-32a2ffbd451b admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:14.186782 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4818d-46e8-43af-8192-32a2ffbd451b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:14.187112 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4818d-46e8-43af-8192-32a2ffbd451b admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:14.187777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4818d-46e8-43af-8192-32a2ffbd451b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:14.189768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e4818d-46e8-43af-8192-32a2ffbd451b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:14.225862 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 897/1797] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:14 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:14.227174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466cfe0-1397-48c5-aea3-83f15b33181f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:14.240232 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466cfe0-1397-48c5-aea3-83f15b33181f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:14.247325 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a3b10-8a9a-47fe-81e5-9d562e1717bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:14.248180 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a3b10-8a9a-47fe-81e5-9d562e1717bc admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:14.248612 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a3b10-8a9a-47fe-81e5-9d562e1717bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:14.249032 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a3b10-8a9a-47fe-81e5-9d562e1717bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:14.249828 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a3b10-8a9a-47fe-81e5-9d562e1717bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:14.252518 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a3b10-8a9a-47fe-81e5-9d562e1717bc admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:14.264729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 901/1798] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:14.269011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4818d-46e8-43af-8192-32a2ffbd451b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:14.298680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4818d-46e8-43af-8192-32a2ffbd451b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:14.305021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e97fda-0d8c-499c-90e4-381c7171c5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:14.305982 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e97fda-0d8c-499c-90e4-381c7171c5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:14.306394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e97fda-0d8c-499c-90e4-381c7171c5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:14.306840 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e97fda-0d8c-499c-90e4-381c7171c5c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:14.308673 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e97fda-0d8c-499c-90e4-381c7171c5c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:14.310619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e97fda-0d8c-499c-90e4-381c7171c5c4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:14.329925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 898/1799] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:14.336958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a3b10-8a9a-47fe-81e5-9d562e1717bc admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:14.357157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a3b10-8a9a-47fe-81e5-9d562e1717bc admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:14.362310 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d77b3-33d1-4cc0-bf92-d73b1c69690f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:14.363033 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d77b3-33d1-4cc0-bf92-d73b1c69690f admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:14.363313 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d77b3-33d1-4cc0-bf92-d73b1c69690f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:14.363588 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d77b3-33d1-4cc0-bf92-d73b1c69690f admin admin] PATH_INFO: `/v3/projects/ba1ddde60ab04563b3601961f3b36eea/users/5696f411c2d640f1a0e1d3309f6f422a/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:14.395648 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d77b3-33d1-4cc0-bf92-d73b1c69690f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba1ddde60ab04563b3601961f3b36eea, user_id=5696f411c2d640f1a0e1d3309f6f422a, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:14.399098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d77b3-33d1-4cc0-bf92-d73b1c69690f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:14.418478 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 902/1800] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:14 2022] PUT /identity/v3/projects/ba1ddde60ab04563b3601961f3b36eea/users/5696f411c2d640f1a0e1d3309f6f422a/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:14.428990 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba1163-83e0-4177-8393-6ee56cfc2c56 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:14.429436 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba1163-83e0-4177-8393-6ee56cfc2c56 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:14.429786 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba1163-83e0-4177-8393-6ee56cfc2c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:14.471315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4fba1163-83e0-4177-8393-6ee56cfc2c56 None None] MFA Rules not processed for user `5696f411c2d640f1a0e1d3309f6f422a`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:14.472598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4fba1163-83e0-4177-8393-6ee56cfc2c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:14.564578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 899/1801] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:14.576937 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d77b3-33d1-4cc0-bf92-d73b1c69690f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:14.578662 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-221d77b3-33d1-4cc0-bf92-d73b1c69690f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:14.673791 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d77b3-33d1-4cc0-bf92-d73b1c69690f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:14.681084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66fe55-f4a5-4bc3-a700-e7cf5cb44548 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:14.681801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66fe55-f4a5-4bc3-a700-e7cf5cb44548 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:14.682938 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66fe55-f4a5-4bc3-a700-e7cf5cb44548 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:14.683293 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66fe55-f4a5-4bc3-a700-e7cf5cb44548 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:14.692570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66fe55-f4a5-4bc3-a700-e7cf5cb44548 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:14.694840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66fe55-f4a5-4bc3-a700-e7cf5cb44548 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:14.712127 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 903/1802] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:14.983218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba1163-83e0-4177-8393-6ee56cfc2c56 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:15.063537 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba1163-83e0-4177-8393-6ee56cfc2c56 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:15.068776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117b890-7559-41cc-93b8-b73f9503ff84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:15.070587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117b890-7559-41cc-93b8-b73f9503ff84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:15.070919 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117b890-7559-41cc-93b8-b73f9503ff84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:15.071710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117b890-7559-41cc-93b8-b73f9503ff84 admin admin] PATH_INFO: `/v3/users/5696f411c2d640f1a0e1d3309f6f422a` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:15.072936 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117b890-7559-41cc-93b8-b73f9503ff84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5696f411c2d640f1a0e1d3309f6f422a)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:15.074979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117b890-7559-41cc-93b8-b73f9503ff84 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:15.093172 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:15.138610 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9117b890-7559-41cc-93b8-b73f9503ff84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5696f411c2d640f1a0e1d3309f6f422a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:15.148708 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9117b890-7559-41cc-93b8-b73f9503ff84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5696f411c2d640f1a0e1d3309f6f422a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:15.149773 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9117b890-7559-41cc-93b8-b73f9503ff84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5696f411c2d640f1a0e1d3309f6f422a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:15.176979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9117b890-7559-41cc-93b8-b73f9503ff84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5696f411c2d640f1a0e1d3309f6f422a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:15.191141 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9117b890-7559-41cc-93b8-b73f9503ff84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5696f411c2d640f1a0e1d3309f6f422a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:15.197902 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 900/1803] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:14 2022] DELETE /identity/v3/users/5696f411c2d640f1a0e1d3309f6f422a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:15.202323 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829063b-a73b-4a58-b749-35d4c3c054c4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:15.202793 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829063b-a73b-4a58-b749-35d4c3c054c4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:15.202914 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829063b-a73b-4a58-b749-35d4c3c054c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:15.224284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9829063b-a73b-4a58-b749-35d4c3c054c4 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:15.226297 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9829063b-a73b-4a58-b749-35d4c3c054c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:15.235519 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:15.295831 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 904/1804] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:15.305640 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117b890-7559-41cc-93b8-b73f9503ff84 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:15.307114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9117b890-7559-41cc-93b8-b73f9503ff84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:15.388861 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117b890-7559-41cc-93b8-b73f9503ff84 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:15.393008 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7ddb2-7f31-4709-bb93-591943e7c106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:15.393631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7ddb2-7f31-4709-bb93-591943e7c106 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:15.393915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7ddb2-7f31-4709-bb93-591943e7c106 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:15.394673 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7ddb2-7f31-4709-bb93-591943e7c106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:15.404562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7ddb2-7f31-4709-bb93-591943e7c106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:15.406004 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7ddb2-7f31-4709-bb93-591943e7c106 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:15.416136 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 901/1805] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:15.805602 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829063b-a73b-4a58-b749-35d4c3c054c4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:15.807125 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9829063b-a73b-4a58-b749-35d4c3c054c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:15.828683 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829063b-a73b-4a58-b749-35d4c3c054c4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:15.835442 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385b73a-aeac-4069-b970-67eed9e23595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:15.836173 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385b73a-aeac-4069-b970-67eed9e23595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:15.836457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385b73a-aeac-4069-b970-67eed9e23595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:15.836855 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385b73a-aeac-4069-b970-67eed9e23595 admin admin] PATH_INFO: `/v3/projects/ba1ddde60ab04563b3601961f3b36eea` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:15.838306 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385b73a-aeac-4069-b970-67eed9e23595 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba1ddde60ab04563b3601961f3b36eea)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:15.840198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385b73a-aeac-4069-b970-67eed9e23595 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:15.894593 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d385b73a-aeac-4069-b970-67eed9e23595 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba1ddde60ab04563b3601961f3b36eea', 'request_id': 'req-d385b73a-aeac-4069-b970-67eed9e23595'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:15.900592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d385b73a-aeac-4069-b970-67eed9e23595 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba1ddde60ab04563b3601961f3b36eea', 'request_id': 'req-d385b73a-aeac-4069-b970-67eed9e23595'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:15.907393 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d385b73a-aeac-4069-b970-67eed9e23595 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba1ddde60ab04563b3601961f3b36eea', 'request_id': 'req-d385b73a-aeac-4069-b970-67eed9e23595'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:15.920199 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d385b73a-aeac-4069-b970-67eed9e23595 admin admin] The token cache is being invalidate because project ba1ddde60ab04563b3601961f3b36eea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:15.920491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d385b73a-aeac-4069-b970-67eed9e23595 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:15.921111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d385b73a-aeac-4069-b970-67eed9e23595 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:15.922601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 905/1806] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:15 2022] DELETE /identity/v3/projects/ba1ddde60ab04563b3601961f3b36eea => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:15.927641 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdc0fb-587e-4f86-8c4e-2397769dccb7 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:15.928455 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdc0fb-587e-4f86-8c4e-2397769dccb7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:15.928731 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdc0fb-587e-4f86-8c4e-2397769dccb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:15.948763 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-cabdc0fb-587e-4f86-8c4e-2397769dccb7 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:15.949940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cabdc0fb-587e-4f86-8c4e-2397769dccb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:16.012279 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 902/1807] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:16.019976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9b4bf-461e-4892-8255-ad0f280c2f9a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:16.020301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9b4bf-461e-4892-8255-ad0f280c2f9a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:16.020584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9b4bf-461e-4892-8255-ad0f280c2f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:16.044083 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a9a9b4bf-461e-4892-8255-ad0f280c2f9a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:16.045402 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a9a9b4bf-461e-4892-8255-ad0f280c2f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:16.055513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 906/1808] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:16.059978 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdc0fb-587e-4f86-8c4e-2397769dccb7 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:16.071002 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdc0fb-587e-4f86-8c4e-2397769dccb7 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:16.076268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751c15e-f682-4f1e-95e5-08f675c259d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:16.076883 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751c15e-f682-4f1e-95e5-08f675c259d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:16.077159 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751c15e-f682-4f1e-95e5-08f675c259d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:16.077428 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751c15e-f682-4f1e-95e5-08f675c259d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:16.078070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c751c15e-f682-4f1e-95e5-08f675c259d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:16.078372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c751c15e-f682-4f1e-95e5-08f675c259d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:16.080079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c751c15e-f682-4f1e-95e5-08f675c259d3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:16.092478 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 903/1809] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:13:16.096986 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9b4bf-461e-4892-8255-ad0f280c2f9a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:16.106561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9b4bf-461e-4892-8255-ad0f280c2f9a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:16.111277 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e85a87-7a06-4de0-9f5f-b618e0261169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:16.111906 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e85a87-7a06-4de0-9f5f-b618e0261169 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:16.112186 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e85a87-7a06-4de0-9f5f-b618e0261169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:16.112500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e85a87-7a06-4de0-9f5f-b618e0261169 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:16.113138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e85a87-7a06-4de0-9f5f-b618e0261169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:16.115045 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e85a87-7a06-4de0-9f5f-b618e0261169 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:16.147575 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 907/1810] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:16 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:16.151690 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751c15e-f682-4f1e-95e5-08f675c259d3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:16.166997 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751c15e-f682-4f1e-95e5-08f675c259d3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:16.198235 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:16.253844 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00412d-3aa6-4b09-bbf4-1c5da6a04039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:16.254799 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00412d-3aa6-4b09-bbf4-1c5da6a04039 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:16.255099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00412d-3aa6-4b09-bbf4-1c5da6a04039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:16.255375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00412d-3aa6-4b09-bbf4-1c5da6a04039 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:16.256090 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac00412d-3aa6-4b09-bbf4-1c5da6a04039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:16.258109 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac00412d-3aa6-4b09-bbf4-1c5da6a04039 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:16.286567 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 904/1811] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:16 2022] POST /identity/v3/users => generated 303 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:16.288271 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e85a87-7a06-4de0-9f5f-b618e0261169 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:16.299269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e85a87-7a06-4de0-9f5f-b618e0261169 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:16.307507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519175f-8398-43b2-b3df-8c02fadb85f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:16.308287 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519175f-8398-43b2-b3df-8c02fadb85f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:16.308672 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519175f-8398-43b2-b3df-8c02fadb85f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:16.309049 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519175f-8398-43b2-b3df-8c02fadb85f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:16.309809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b519175f-8398-43b2-b3df-8c02fadb85f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:16.311954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b519175f-8398-43b2-b3df-8c02fadb85f7 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:16.323632 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 908/1812] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:13:16.327880 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00412d-3aa6-4b09-bbf4-1c5da6a04039 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:16.339635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00412d-3aa6-4b09-bbf4-1c5da6a04039 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:16.345452 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d8904-2b4d-4d46-b0e7-f6e60ee570f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:16.346342 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d8904-2b4d-4d46-b0e7-f6e60ee570f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:16.346701 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d8904-2b4d-4d46-b0e7-f6e60ee570f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:16.347089 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d8904-2b4d-4d46-b0e7-f6e60ee570f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:16.347964 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993d8904-2b4d-4d46-b0e7-f6e60ee570f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:16.349668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993d8904-2b4d-4d46-b0e7-f6e60ee570f5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:16.369134 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 905/1813] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:13:16.370212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519175f-8398-43b2-b3df-8c02fadb85f7 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:16.383151 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519175f-8398-43b2-b3df-8c02fadb85f7 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:16.388225 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ddf5b-b92c-4d2e-9f5f-7af8e29bc06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:16.388898 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ddf5b-b92c-4d2e-9f5f-7af8e29bc06a admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:16.389178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ddf5b-b92c-4d2e-9f5f-7af8e29bc06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:16.389452 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ddf5b-b92c-4d2e-9f5f-7af8e29bc06a admin admin] PATH_INFO: `/v3/projects/3ffbcac907054f608ee7eaa6fddf152b/users/21dc0c4a458749bebfacc0436a242136/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:16.406657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ddf5b-b92c-4d2e-9f5f-7af8e29bc06a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ffbcac907054f608ee7eaa6fddf152b, user_id=21dc0c4a458749bebfacc0436a242136, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:16.409431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ddf5b-b92c-4d2e-9f5f-7af8e29bc06a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:16.424501 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 909/1814] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:16 2022] PUT /identity/v3/projects/3ffbcac907054f608ee7eaa6fddf152b/users/21dc0c4a458749bebfacc0436a242136/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:16.430014 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca6bfa-5538-4754-bf82-9ad238202592 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:16.430323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca6bfa-5538-4754-bf82-9ad238202592 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:16.430580 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca6bfa-5538-4754-bf82-9ad238202592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:16.452324 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-79ca6bfa-5538-4754-bf82-9ad238202592 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:16.453565 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-79ca6bfa-5538-4754-bf82-9ad238202592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:16.515501 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 906/1815] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:16.526448 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ddf5b-b92c-4d2e-9f5f-7af8e29bc06a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:16.528547 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e84ddf5b-b92c-4d2e-9f5f-7af8e29bc06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:16.572862 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:16.594219 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ddf5b-b92c-4d2e-9f5f-7af8e29bc06a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:16.597977 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e5651-3684-4b9a-81ce-fd60ce516ad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:16.598591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e5651-3684-4b9a-81ce-fd60ce516ad8 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:16.598889 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e5651-3684-4b9a-81ce-fd60ce516ad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:16.599176 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e5651-3684-4b9a-81ce-fd60ce516ad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:16.608735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e5651-3684-4b9a-81ce-fd60ce516ad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:16.610405 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e5651-3684-4b9a-81ce-fd60ce516ad8 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:16.621315 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 910/1816] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:16.992513 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca6bfa-5538-4754-bf82-9ad238202592 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:16.994085 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-79ca6bfa-5538-4754-bf82-9ad238202592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:17.008875 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca6bfa-5538-4754-bf82-9ad238202592 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:17.015431 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb3dbe-0ab3-490a-bbec-10288ddba4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:17.019306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb3dbe-0ab3-490a-bbec-10288ddba4ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:17.019635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb3dbe-0ab3-490a-bbec-10288ddba4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:17.020940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb3dbe-0ab3-490a-bbec-10288ddba4ae admin admin] PATH_INFO: `/v3/users/eec2a2cdcafd40cd8b8e6f80c3c54191` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:17.023978 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb3dbe-0ab3-490a-bbec-10288ddba4ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=eec2a2cdcafd40cd8b8e6f80c3c54191)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:17.026219 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb3dbe-0ab3-490a-bbec-10288ddba4ae admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:17.101807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1edb3dbe-0ab3-490a-bbec-10288ddba4ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eec2a2cdcafd40cd8b8e6f80c3c54191'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:17.113037 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1edb3dbe-0ab3-490a-bbec-10288ddba4ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eec2a2cdcafd40cd8b8e6f80c3c54191'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:17.113833 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1edb3dbe-0ab3-490a-bbec-10288ddba4ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eec2a2cdcafd40cd8b8e6f80c3c54191'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:17.124740 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1edb3dbe-0ab3-490a-bbec-10288ddba4ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eec2a2cdcafd40cd8b8e6f80c3c54191'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:17.146640 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1edb3dbe-0ab3-490a-bbec-10288ddba4ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eec2a2cdcafd40cd8b8e6f80c3c54191'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:17.154667 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 907/1817] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:16 2022] DELETE /identity/v3/users/eec2a2cdcafd40cd8b8e6f80c3c54191 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:17.159026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc40e6-8ba5-4b55-b6df-0f2d38290c8b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:17.160234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc40e6-8ba5-4b55-b6df-0f2d38290c8b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:17.160234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc40e6-8ba5-4b55-b6df-0f2d38290c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:17.181133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9ebc40e6-8ba5-4b55-b6df-0f2d38290c8b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:17.182380 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9ebc40e6-8ba5-4b55-b6df-0f2d38290c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:17.260900 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 911/1818] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:17.267285 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb3dbe-0ab3-490a-bbec-10288ddba4ae admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:17.268985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1edb3dbe-0ab3-490a-bbec-10288ddba4ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:17.291014 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:17.344863 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb3dbe-0ab3-490a-bbec-10288ddba4ae admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:17.348863 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b760c0c-843f-4ec4-be96-85b0d4ddd627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:17.349508 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b760c0c-843f-4ec4-be96-85b0d4ddd627 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:17.349845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b760c0c-843f-4ec4-be96-85b0d4ddd627 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:17.350117 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b760c0c-843f-4ec4-be96-85b0d4ddd627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:17.360077 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b760c0c-843f-4ec4-be96-85b0d4ddd627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:17.361486 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b760c0c-843f-4ec4-be96-85b0d4ddd627 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:17.376989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 908/1819] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:13:17.652148 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc40e6-8ba5-4b55-b6df-0f2d38290c8b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:17.653988 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9ebc40e6-8ba5-4b55-b6df-0f2d38290c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:17.667970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc40e6-8ba5-4b55-b6df-0f2d38290c8b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:17.698085 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc489b6-bb8b-4721-adb2-b4f6ed635ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:17.699121 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc489b6-bb8b-4721-adb2-b4f6ed635ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:17.699886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc489b6-bb8b-4721-adb2-b4f6ed635ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:17.700330 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc489b6-bb8b-4721-adb2-b4f6ed635ac8 admin admin] PATH_INFO: `/v3/projects/fc4fa086ae1647e48281586eba4c06e2` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:17.701926 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc489b6-bb8b-4721-adb2-b4f6ed635ac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc4fa086ae1647e48281586eba4c06e2)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:17.706024 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc489b6-bb8b-4721-adb2-b4f6ed635ac8 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:17.766961 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5cc489b6-bb8b-4721-adb2-b4f6ed635ac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc4fa086ae1647e48281586eba4c06e2', 'request_id': 'req-5cc489b6-bb8b-4721-adb2-b4f6ed635ac8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:17.775116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5cc489b6-bb8b-4721-adb2-b4f6ed635ac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc4fa086ae1647e48281586eba4c06e2', 'request_id': 'req-5cc489b6-bb8b-4721-adb2-b4f6ed635ac8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:17.783679 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5cc489b6-bb8b-4721-adb2-b4f6ed635ac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc4fa086ae1647e48281586eba4c06e2', 'request_id': 'req-5cc489b6-bb8b-4721-adb2-b4f6ed635ac8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:17.806979 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5cc489b6-bb8b-4721-adb2-b4f6ed635ac8 admin admin] The token cache is being invalidate because project fc4fa086ae1647e48281586eba4c06e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:17.807407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5cc489b6-bb8b-4721-adb2-b4f6ed635ac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:17.808165 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5cc489b6-bb8b-4721-adb2-b4f6ed635ac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:17.810127 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 912/1820] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:17 2022] DELETE /identity/v3/projects/fc4fa086ae1647e48281586eba4c06e2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:17.813883 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b760c0c-843f-4ec4-be96-85b0d4ddd627 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:17.830567 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b760c0c-843f-4ec4-be96-85b0d4ddd627 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:17.863788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21502836-c261-434e-84a6-b424c36ed127 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:17.863788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21502836-c261-434e-84a6-b424c36ed127 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:17.863788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21502836-c261-434e-84a6-b424c36ed127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:17.917798 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-21502836-c261-434e-84a6-b424c36ed127 None None] MFA Rules not processed for user `21dc0c4a458749bebfacc0436a242136`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:17.919077 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-21502836-c261-434e-84a6-b424c36ed127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:17.932931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff86a1-4513-40e8-9761-5c452da3fced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:17.933845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff86a1-4513-40e8-9761-5c452da3fced admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:17.934297 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff86a1-4513-40e8-9761-5c452da3fced admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:17.934574 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff86a1-4513-40e8-9761-5c452da3fced admin admin] PATH_INFO: `/v3/users/7194a60f95e4409f9847694a21d12721` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:17.937826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff86a1-4513-40e8-9761-5c452da3fced admin admin] RBAC: Authorizing `identity:delete_user(user_id=7194a60f95e4409f9847694a21d12721)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:17.942488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff86a1-4513-40e8-9761-5c452da3fced admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:18.001908 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:18.039978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 913/1821] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:17 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:13:18.041866 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-23ff86a1-4513-40e8-9761-5c452da3fced admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7194a60f95e4409f9847694a21d12721'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:18.043452 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21502836-c261-434e-84a6-b424c36ed127 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:18.046096 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-21502836-c261-434e-84a6-b424c36ed127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:18.062407 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-23ff86a1-4513-40e8-9761-5c452da3fced admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7194a60f95e4409f9847694a21d12721'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:18.063271 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-23ff86a1-4513-40e8-9761-5c452da3fced admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7194a60f95e4409f9847694a21d12721'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:18.070811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21502836-c261-434e-84a6-b424c36ed127 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:18.079833 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd265cc-8527-4a8e-8634-3919fff843f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:18.080708 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd265cc-8527-4a8e-8634-3919fff843f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:18.081161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd265cc-8527-4a8e-8634-3919fff843f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:18.082500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd265cc-8527-4a8e-8634-3919fff843f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:18.083198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd265cc-8527-4a8e-8634-3919fff843f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:18.085504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd265cc-8527-4a8e-8634-3919fff843f4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:18.086670 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-23ff86a1-4513-40e8-9761-5c452da3fced admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7194a60f95e4409f9847694a21d12721'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:18.110347 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-23ff86a1-4513-40e8-9761-5c452da3fced admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7194a60f95e4409f9847694a21d12721'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:18.127404 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 909/1822] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:17 2022] DELETE /identity/v3/users/7194a60f95e4409f9847694a21d12721 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:18.149348 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 914/1823] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:18 2022] POST /identity/v3/projects => generated 401 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:18.153267 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff86a1-4513-40e8-9761-5c452da3fced admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:18.154800 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-23ff86a1-4513-40e8-9761-5c452da3fced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:18.201514 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:18.246284 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff86a1-4513-40e8-9761-5c452da3fced admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:18.319308 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0156fe5-0079-412e-b41a-475200a0aa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:18.320052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0156fe5-0079-412e-b41a-475200a0aa3f admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:18.320238 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0156fe5-0079-412e-b41a-475200a0aa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:18.320564 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0156fe5-0079-412e-b41a-475200a0aa3f admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:18.321252 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0156fe5-0079-412e-b41a-475200a0aa3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:18.323222 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0156fe5-0079-412e-b41a-475200a0aa3f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:18.349189 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 910/1824] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:18 2022] POST /identity/v3/users => generated 307 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:18.351376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd265cc-8527-4a8e-8634-3919fff843f4 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:18.361694 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd265cc-8527-4a8e-8634-3919fff843f4 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:18.366698 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31240057-73b1-41f5-9138-455635c5a46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:18.367455 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31240057-73b1-41f5-9138-455635c5a46d admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:18.367875 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31240057-73b1-41f5-9138-455635c5a46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:18.368326 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31240057-73b1-41f5-9138-455635c5a46d admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:18.369082 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31240057-73b1-41f5-9138-455635c5a46d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:18.372041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31240057-73b1-41f5-9138-455635c5a46d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:18.390903 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 915/1825] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:18.395219 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0156fe5-0079-412e-b41a-475200a0aa3f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:18.409655 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0156fe5-0079-412e-b41a-475200a0aa3f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:18.431655 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67340a8-9418-463f-9957-207612336cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:18.432426 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67340a8-9418-463f-9957-207612336cc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:18.432718 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67340a8-9418-463f-9957-207612336cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:18.433311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67340a8-9418-463f-9957-207612336cc8 admin admin] PATH_INFO: `/v3/projects/1d4ef781b1084a79be8fc5ae480b35db/users/8180371c2a33413883a71f426376268d/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:18.446020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31240057-73b1-41f5-9138-455635c5a46d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:18.447833 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-31240057-73b1-41f5-9138-455635c5a46d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:18.463614 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67340a8-9418-463f-9957-207612336cc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4ef781b1084a79be8fc5ae480b35db, user_id=8180371c2a33413883a71f426376268d, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:18.468019 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67340a8-9418-463f-9957-207612336cc8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:18.470339 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31240057-73b1-41f5-9138-455635c5a46d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:18.476239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06980d-589a-4bb5-b46e-e883ab063cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:18.477028 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06980d-589a-4bb5-b46e-e883ab063cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:18.477290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06980d-589a-4bb5-b46e-e883ab063cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:18.477593 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06980d-589a-4bb5-b46e-e883ab063cac admin admin] PATH_INFO: `/v3/projects/b883e063e75a4a82929da6bd4a713124` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:18.479314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06980d-589a-4bb5-b46e-e883ab063cac admin admin] RBAC: Authorizing `identity:delete_project(project_id=b883e063e75a4a82929da6bd4a713124)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:18.481196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf06980d-589a-4bb5-b46e-e883ab063cac admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:18.489129 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 911/1826] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:18 2022] PUT /identity/v3/projects/1d4ef781b1084a79be8fc5ae480b35db/users/8180371c2a33413883a71f426376268d/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:13:18.491021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67340a8-9418-463f-9957-207612336cc8 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:18.537923 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf06980d-589a-4bb5-b46e-e883ab063cac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b883e063e75a4a82929da6bd4a713124', 'request_id': 'req-cf06980d-589a-4bb5-b46e-e883ab063cac'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:18.544735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf06980d-589a-4bb5-b46e-e883ab063cac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b883e063e75a4a82929da6bd4a713124', 'request_id': 'req-cf06980d-589a-4bb5-b46e-e883ab063cac'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:18.552747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf06980d-589a-4bb5-b46e-e883ab063cac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b883e063e75a4a82929da6bd4a713124', 'request_id': 'req-cf06980d-589a-4bb5-b46e-e883ab063cac'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:18.566033 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67340a8-9418-463f-9957-207612336cc8 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:18.569965 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf06980d-589a-4bb5-b46e-e883ab063cac admin admin] The token cache is being invalidate because project b883e063e75a4a82929da6bd4a713124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:18.570733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf06980d-589a-4bb5-b46e-e883ab063cac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:18.571595 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf06980d-589a-4bb5-b46e-e883ab063cac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:18.575081 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 916/1827] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:18 2022] DELETE /identity/v3/projects/b883e063e75a4a82929da6bd4a713124 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:18.579284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06980d-589a-4bb5-b46e-e883ab063cac admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:18.583264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cf06980d-589a-4bb5-b46e-e883ab063cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:18.635012 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:18.645694 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d64049-7aa6-497f-8b08-ae471dc1d826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:18.646318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d64049-7aa6-497f-8b08-ae471dc1d826 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:18.646679 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d64049-7aa6-497f-8b08-ae471dc1d826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:18.647027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d64049-7aa6-497f-8b08-ae471dc1d826 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:18.647855 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d64049-7aa6-497f-8b08-ae471dc1d826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:18.650344 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d64049-7aa6-497f-8b08-ae471dc1d826 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:18.653383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf06980d-589a-4bb5-b46e-e883ab063cac admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:18.658554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e34cd-31c9-4d9e-87ad-33414e308cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:18.659233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e34cd-31c9-4d9e-87ad-33414e308cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:18.659517 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e34cd-31c9-4d9e-87ad-33414e308cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:18.659907 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e34cd-31c9-4d9e-87ad-33414e308cc4 admin admin] PATH_INFO: `/v3/users/223ab369468743699d84b4d0a8741bb8` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:18.661061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e34cd-31c9-4d9e-87ad-33414e308cc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=223ab369468743699d84b4d0a8741bb8)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:18.661318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 912/1828] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:18.664714 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8e34cd-31c9-4d9e-87ad-33414e308cc4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:18.666175 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d64049-7aa6-497f-8b08-ae471dc1d826 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:18.668449 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e9d64049-7aa6-497f-8b08-ae471dc1d826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:18.683807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d64049-7aa6-497f-8b08-ae471dc1d826 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:18.691534 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bf572-fffd-4c8e-ba70-4436bd4c86f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:18.692337 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bf572-fffd-4c8e-ba70-4436bd4c86f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:18.692508 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bf572-fffd-4c8e-ba70-4436bd4c86f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:18.692793 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bf572-fffd-4c8e-ba70-4436bd4c86f2 admin admin] PATH_INFO: `/v3/projects/1d4ef781b1084a79be8fc5ae480b35db/users/8180371c2a33413883a71f426376268d/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:18.695181 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bf572-fffd-4c8e-ba70-4436bd4c86f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d4ef781b1084a79be8fc5ae480b35db, user_id=8180371c2a33413883a71f426376268d, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:18.698107 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bf572-fffd-4c8e-ba70-4436bd4c86f2 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:18.710588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 913/1829] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:18 2022] PUT /identity/v3/projects/1d4ef781b1084a79be8fc5ae480b35db/users/8180371c2a33413883a71f426376268d/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:13:18.723238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:18.727090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7d8e34cd-31c9-4d9e-87ad-33414e308cc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '223ab369468743699d84b4d0a8741bb8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:18.735320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7d8e34cd-31c9-4d9e-87ad-33414e308cc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '223ab369468743699d84b4d0a8741bb8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:18.736652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7d8e34cd-31c9-4d9e-87ad-33414e308cc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '223ab369468743699d84b4d0a8741bb8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:18.747758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7d8e34cd-31c9-4d9e-87ad-33414e308cc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '223ab369468743699d84b4d0a8741bb8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:18.778202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7d8e34cd-31c9-4d9e-87ad-33414e308cc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '223ab369468743699d84b4d0a8741bb8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:18.785174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 917/1830] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:18 2022] DELETE /identity/v3/users/223ab369468743699d84b4d0a8741bb8 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:13:19.436584 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bf572-fffd-4c8e-ba70-4436bd4c86f2 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:19.438709 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-098bf572-fffd-4c8e-ba70-4436bd4c86f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:19.529486 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bf572-fffd-4c8e-ba70-4436bd4c86f2 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:19.589203 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de33fd3-bb63-447e-a80a-a6d6faf4352d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:19.590483 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de33fd3-bb63-447e-a80a-a6d6faf4352d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:19.590483 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de33fd3-bb63-447e-a80a-a6d6faf4352d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:19.590793 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de33fd3-bb63-447e-a80a-a6d6faf4352d admin admin] PATH_INFO: `/v3/projects/0dc8dfcd2640459e9b9b18b3df82db38` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:19.591873 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de33fd3-bb63-447e-a80a-a6d6faf4352d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dc8dfcd2640459e9b9b18b3df82db38)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:19.594078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de33fd3-bb63-447e-a80a-a6d6faf4352d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:19.646697 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6de33fd3-bb63-447e-a80a-a6d6faf4352d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dc8dfcd2640459e9b9b18b3df82db38', 'request_id': 'req-6de33fd3-bb63-447e-a80a-a6d6faf4352d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:19.653689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6de33fd3-bb63-447e-a80a-a6d6faf4352d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dc8dfcd2640459e9b9b18b3df82db38', 'request_id': 'req-6de33fd3-bb63-447e-a80a-a6d6faf4352d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:19.660313 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6de33fd3-bb63-447e-a80a-a6d6faf4352d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dc8dfcd2640459e9b9b18b3df82db38', 'request_id': 'req-6de33fd3-bb63-447e-a80a-a6d6faf4352d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:19.687002 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6de33fd3-bb63-447e-a80a-a6d6faf4352d admin admin] The token cache is being invalidate because project 0dc8dfcd2640459e9b9b18b3df82db38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:19.687788 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6de33fd3-bb63-447e-a80a-a6d6faf4352d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:19.688921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-6de33fd3-bb63-447e-a80a-a6d6faf4352d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:19.691002 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 914/1831] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:19 2022] DELETE /identity/v3/projects/0dc8dfcd2640459e9b9b18b3df82db38 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:19.697795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8fe28-0219-4c52-b7ce-851af7579a4c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:19.698304 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cb9f4-0690-4f55-8de1-1805eb79e5eb None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:19.698410 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8fe28-0219-4c52-b7ce-851af7579a4c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:19.698514 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cb9f4-0690-4f55-8de1-1805eb79e5eb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:19.698608 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8fe28-0219-4c52-b7ce-851af7579a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:19.698700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cb9f4-0690-4f55-8de1-1805eb79e5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:19.718850 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-dee8fe28-0219-4c52-b7ce-851af7579a4c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:19.721113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dee8fe28-0219-4c52-b7ce-851af7579a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:19.728501 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3f9cb9f4-0690-4f55-8de1-1805eb79e5eb None None] MFA Rules not processed for user `8180371c2a33413883a71f426376268d`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:19.730161 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3f9cb9f4-0690-4f55-8de1-1805eb79e5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:19.734260 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:19.825887 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 915/1832] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:19 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:13:19.826116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 918/1833] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:19.829031 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2116df-ee49-46d3-956c-6933400dbad5 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:19.829319 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2116df-ee49-46d3-956c-6933400dbad5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:19.829596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2116df-ee49-46d3-956c-6933400dbad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:19.852833 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1e2116df-ee49-46d3-956c-6933400dbad5 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:19.854034 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1e2116df-ee49-46d3-956c-6933400dbad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:19.867402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 916/1834] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:19.871890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8fe28-0219-4c52-b7ce-851af7579a4c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:19.890167 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8fe28-0219-4c52-b7ce-851af7579a4c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:19.898056 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835b75b-2f07-4562-a4f2-dbde4a91608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:19.899109 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835b75b-2f07-4562-a4f2-dbde4a91608e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:19.899652 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835b75b-2f07-4562-a4f2-dbde4a91608e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:19.900168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835b75b-2f07-4562-a4f2-dbde4a91608e admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:19.901112 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835b75b-2f07-4562-a4f2-dbde4a91608e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:19.902192 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835b75b-2f07-4562-a4f2-dbde4a91608e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:19.904331 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835b75b-2f07-4562-a4f2-dbde4a91608e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:19.922669 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 919/1835] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:13:19.929146 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2116df-ee49-46d3-956c-6933400dbad5 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:19.942251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2116df-ee49-46d3-956c-6933400dbad5 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:19.948754 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1041e-6ff5-465a-a4a2-0f76b446a4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:19.949470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1041e-6ff5-465a-a4a2-0f76b446a4f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:19.950065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1041e-6ff5-465a-a4a2-0f76b446a4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:19.950065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1041e-6ff5-465a-a4a2-0f76b446a4f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:19.951224 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1041e-6ff5-465a-a4a2-0f76b446a4f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:19.953041 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1041e-6ff5-465a-a4a2-0f76b446a4f2 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:19.991836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 917/1836] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:19 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:13:19.996073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835b75b-2f07-4562-a4f2-dbde4a91608e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:20.007090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835b75b-2f07-4562-a4f2-dbde4a91608e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:20.067671 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0fc07-857b-4273-9468-4f6fe232c8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:20.069279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0fc07-857b-4273-9468-4f6fe232c8da admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:20.069920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0fc07-857b-4273-9468-4f6fe232c8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:20.070251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba0fc07-857b-4273-9468-4f6fe232c8da admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:20.070949 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0fc07-857b-4273-9468-4f6fe232c8da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:20.072976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba0fc07-857b-4273-9468-4f6fe232c8da admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:20.099136 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 920/1837] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:19 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:13:20.104197 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1041e-6ff5-465a-a4a2-0f76b446a4f2 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:20.114783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1041e-6ff5-465a-a4a2-0f76b446a4f2 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:20.119575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3c66e-d23c-4980-9c74-9172756399c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:20.120185 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3c66e-d23c-4980-9c74-9172756399c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:20.120490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3c66e-d23c-4980-9c74-9172756399c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:20.120763 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3c66e-d23c-4980-9c74-9172756399c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:20.121401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3c66e-d23c-4980-9c74-9172756399c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:20.123036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3c66e-d23c-4980-9c74-9172756399c8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:20.135034 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 918/1838] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:20.139930 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0fc07-857b-4273-9468-4f6fe232c8da admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:20.150527 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba0fc07-857b-4273-9468-4f6fe232c8da admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:20.156369 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a0015-0918-4127-9787-ac3278aee3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:20.156639 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a0015-0918-4127-9787-ac3278aee3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:20.156931 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a0015-0918-4127-9787-ac3278aee3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:20.157206 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a0015-0918-4127-9787-ac3278aee3e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:20.158107 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a0015-0918-4127-9787-ac3278aee3e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:20.159479 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a0015-0918-4127-9787-ac3278aee3e0 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:20.171421 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 921/1839] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:20.175654 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3c66e-d23c-4980-9c74-9172756399c8 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:20.186709 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3c66e-d23c-4980-9c74-9172756399c8 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:20.191694 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726b988-73e0-4ae3-84b3-416eb61c9f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:20.192401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726b988-73e0-4ae3-84b3-416eb61c9f56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:20.192625 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726b988-73e0-4ae3-84b3-416eb61c9f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:20.192898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726b988-73e0-4ae3-84b3-416eb61c9f56 admin admin] PATH_INFO: `/v3/projects/4b832feb5f7644a9b93ce9e99af3ef11/users/15739afe0c9341de996c6f963b08e52a/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:20.208784 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726b988-73e0-4ae3-84b3-416eb61c9f56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b832feb5f7644a9b93ce9e99af3ef11, user_id=15739afe0c9341de996c6f963b08e52a, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:20.211722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726b988-73e0-4ae3-84b3-416eb61c9f56 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:20.233353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 919/1840] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:20 2022] PUT /identity/v3/projects/4b832feb5f7644a9b93ce9e99af3ef11/users/15739afe0c9341de996c6f963b08e52a/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:20.245054 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b1ea4-de4d-4443-82eb-6c71907aa649 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:20.245358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b1ea4-de4d-4443-82eb-6c71907aa649 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:20.245615 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b1ea4-de4d-4443-82eb-6c71907aa649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:20.273881 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f69b1ea4-de4d-4443-82eb-6c71907aa649 None None] MFA Rules not processed for user `15739afe0c9341de996c6f963b08e52a`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:20.280568 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f69b1ea4-de4d-4443-82eb-6c71907aa649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:20.295955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:20.365034 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 922/1841] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:20.375861 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726b988-73e0-4ae3-84b3-416eb61c9f56 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:20.377779 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f726b988-73e0-4ae3-84b3-416eb61c9f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:20.464701 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726b988-73e0-4ae3-84b3-416eb61c9f56 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:20.473789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ea065-b7f0-48c6-833c-7ce59f821272 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:20.474906 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ea065-b7f0-48c6-833c-7ce59f821272 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:20.474906 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ea065-b7f0-48c6-833c-7ce59f821272 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:20.475740 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ea065-b7f0-48c6-833c-7ce59f821272 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:20.486168 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ea065-b7f0-48c6-833c-7ce59f821272 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:20.488339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ea065-b7f0-48c6-833c-7ce59f821272 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:20.500389 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 920/1842] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:21.233132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b1ea4-de4d-4443-82eb-6c71907aa649 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:21.234668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f69b1ea4-de4d-4443-82eb-6c71907aa649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:21.303537 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b1ea4-de4d-4443-82eb-6c71907aa649 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:21.309177 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b03a4-81ff-4dd5-b30a-ed7fb4333264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:21.309829 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b03a4-81ff-4dd5-b30a-ed7fb4333264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:21.310115 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b03a4-81ff-4dd5-b30a-ed7fb4333264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:21.310395 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b03a4-81ff-4dd5-b30a-ed7fb4333264 admin admin] PATH_INFO: `/v3/users/21dc0c4a458749bebfacc0436a242136` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:21.311572 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b03a4-81ff-4dd5-b30a-ed7fb4333264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21dc0c4a458749bebfacc0436a242136)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:21.314414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b03a4-81ff-4dd5-b30a-ed7fb4333264 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:21.389212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b50b03a4-81ff-4dd5-b30a-ed7fb4333264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21dc0c4a458749bebfacc0436a242136'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:21.396707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b50b03a4-81ff-4dd5-b30a-ed7fb4333264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21dc0c4a458749bebfacc0436a242136'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:21.397721 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b50b03a4-81ff-4dd5-b30a-ed7fb4333264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21dc0c4a458749bebfacc0436a242136'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:21.406765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b50b03a4-81ff-4dd5-b30a-ed7fb4333264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21dc0c4a458749bebfacc0436a242136'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:21.422792 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b50b03a4-81ff-4dd5-b30a-ed7fb4333264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21dc0c4a458749bebfacc0436a242136'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:21.432169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 923/1843] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:21 2022] DELETE /identity/v3/users/21dc0c4a458749bebfacc0436a242136 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:21.652559 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ea065-b7f0-48c6-833c-7ce59f821272 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:21.676463 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ea065-b7f0-48c6-833c-7ce59f821272 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:21.732081 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:21.752730 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba723546-ab12-4627-a4fb-9c10892b3a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:21.753363 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba723546-ab12-4627-a4fb-9c10892b3a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:21.753641 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba723546-ab12-4627-a4fb-9c10892b3a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:21.753915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba723546-ab12-4627-a4fb-9c10892b3a17 admin admin] PATH_INFO: `/v3/projects/3ffbcac907054f608ee7eaa6fddf152b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:21.755061 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba723546-ab12-4627-a4fb-9c10892b3a17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ffbcac907054f608ee7eaa6fddf152b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:21.757059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba723546-ab12-4627-a4fb-9c10892b3a17 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:21.804243 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ba723546-ab12-4627-a4fb-9c10892b3a17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ffbcac907054f608ee7eaa6fddf152b', 'request_id': 'req-ba723546-ab12-4627-a4fb-9c10892b3a17'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:21.809998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ba723546-ab12-4627-a4fb-9c10892b3a17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ffbcac907054f608ee7eaa6fddf152b', 'request_id': 'req-ba723546-ab12-4627-a4fb-9c10892b3a17'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:21.817525 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ba723546-ab12-4627-a4fb-9c10892b3a17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ffbcac907054f608ee7eaa6fddf152b', 'request_id': 'req-ba723546-ab12-4627-a4fb-9c10892b3a17'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:21.832812 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ba723546-ab12-4627-a4fb-9c10892b3a17 admin admin] The token cache is being invalidate because project 3ffbcac907054f608ee7eaa6fddf152b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:21.833129 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ba723546-ab12-4627-a4fb-9c10892b3a17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:21.833789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ba723546-ab12-4627-a4fb-9c10892b3a17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:21.835957 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 921/1844] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:21 2022] DELETE /identity/v3/projects/3ffbcac907054f608ee7eaa6fddf152b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:21.840360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b03a4-81ff-4dd5-b30a-ed7fb4333264 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:21.841809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b50b03a4-81ff-4dd5-b30a-ed7fb4333264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:21.856231 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b03a4-81ff-4dd5-b30a-ed7fb4333264 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:21.892997 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:21.922951 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdb4fd-2b07-4094-a3cf-d0dba49f7ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:21.923642 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdb4fd-2b07-4094-a3cf-d0dba49f7ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:21.923921 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdb4fd-2b07-4094-a3cf-d0dba49f7ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:21.924192 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdb4fd-2b07-4094-a3cf-d0dba49f7ea2 admin admin] PATH_INFO: `/v3/users/8180371c2a33413883a71f426376268d` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:21.925885 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debdb4fd-2b07-4094-a3cf-d0dba49f7ea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8180371c2a33413883a71f426376268d)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:21.927914 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debdb4fd-2b07-4094-a3cf-d0dba49f7ea2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:21.985913 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-debdb4fd-2b07-4094-a3cf-d0dba49f7ea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8180371c2a33413883a71f426376268d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:21.993507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-debdb4fd-2b07-4094-a3cf-d0dba49f7ea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8180371c2a33413883a71f426376268d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:21.994644 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-debdb4fd-2b07-4094-a3cf-d0dba49f7ea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8180371c2a33413883a71f426376268d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:22.003504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-debdb4fd-2b07-4094-a3cf-d0dba49f7ea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8180371c2a33413883a71f426376268d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:22.020195 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-debdb4fd-2b07-4094-a3cf-d0dba49f7ea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8180371c2a33413883a71f426376268d'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:22.034891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 924/1845] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:21 2022] DELETE /identity/v3/users/8180371c2a33413883a71f426376268d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:22.247648 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba723546-ab12-4627-a4fb-9c10892b3a17 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:22.249679 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ba723546-ab12-4627-a4fb-9c10892b3a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:22.283382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba723546-ab12-4627-a4fb-9c10892b3a17 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:22.391795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce71eb-139c-4d31-9198-ab5e03e378c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:22.392825 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce71eb-139c-4d31-9198-ab5e03e378c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:22.393274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce71eb-139c-4d31-9198-ab5e03e378c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:22.393837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce71eb-139c-4d31-9198-ab5e03e378c3 admin admin] PATH_INFO: `/v3/projects/1d4ef781b1084a79be8fc5ae480b35db` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:22.395657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce71eb-139c-4d31-9198-ab5e03e378c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d4ef781b1084a79be8fc5ae480b35db)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:22.399109 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce71eb-139c-4d31-9198-ab5e03e378c3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:22.458041 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-dbce71eb-139c-4d31-9198-ab5e03e378c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d4ef781b1084a79be8fc5ae480b35db', 'request_id': 'req-dbce71eb-139c-4d31-9198-ab5e03e378c3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:22.466069 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-dbce71eb-139c-4d31-9198-ab5e03e378c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d4ef781b1084a79be8fc5ae480b35db', 'request_id': 'req-dbce71eb-139c-4d31-9198-ab5e03e378c3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:22.475516 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-dbce71eb-139c-4d31-9198-ab5e03e378c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d4ef781b1084a79be8fc5ae480b35db', 'request_id': 'req-dbce71eb-139c-4d31-9198-ab5e03e378c3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:22.505078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:22.510208 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-dbce71eb-139c-4d31-9198-ab5e03e378c3 admin admin] The token cache is being invalidate because project 1d4ef781b1084a79be8fc5ae480b35db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:22.510707 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-dbce71eb-139c-4d31-9198-ab5e03e378c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:22.511785 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-dbce71eb-139c-4d31-9198-ab5e03e378c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:22.513680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 922/1846] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:22 2022] DELETE /identity/v3/projects/1d4ef781b1084a79be8fc5ae480b35db => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:22.519102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7a070-a006-4687-abfe-28e63253deb5 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:22.519994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7a070-a006-4687-abfe-28e63253deb5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:22.520464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7a070-a006-4687-abfe-28e63253deb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:22.542495 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-fce7a070-a006-4687-abfe-28e63253deb5 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:22.543799 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fce7a070-a006-4687-abfe-28e63253deb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:22.618078 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 925/1847] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:22.631153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191167f-ecc2-4ad7-bc4c-8367e3faad47 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:22.631537 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191167f-ecc2-4ad7-bc4c-8367e3faad47 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:22.631822 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191167f-ecc2-4ad7-bc4c-8367e3faad47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:22.685413 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f191167f-ecc2-4ad7-bc4c-8367e3faad47 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:22.689224 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f191167f-ecc2-4ad7-bc4c-8367e3faad47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:22.705281 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 923/1848] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:22.712221 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7a070-a006-4687-abfe-28e63253deb5 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:22.739102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7a070-a006-4687-abfe-28e63253deb5 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:22.755326 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f469f71-a2dd-45d2-8b89-a06b2a009e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:22.756529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f469f71-a2dd-45d2-8b89-a06b2a009e3e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:22.757036 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f469f71-a2dd-45d2-8b89-a06b2a009e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:22.757462 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f469f71-a2dd-45d2-8b89-a06b2a009e3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:22.759035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f469f71-a2dd-45d2-8b89-a06b2a009e3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:22.759035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f469f71-a2dd-45d2-8b89-a06b2a009e3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:22.761561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f469f71-a2dd-45d2-8b89-a06b2a009e3e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:22.797356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 926/1849] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:13:22.804544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191167f-ecc2-4ad7-bc4c-8367e3faad47 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:22.824704 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191167f-ecc2-4ad7-bc4c-8367e3faad47 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:22.833499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be51ca2-9244-416b-a2a7-445528aae9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:22.835526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be51ca2-9244-416b-a2a7-445528aae9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:22.835902 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be51ca2-9244-416b-a2a7-445528aae9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:22.836252 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be51ca2-9244-416b-a2a7-445528aae9c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:22.837260 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be51ca2-9244-416b-a2a7-445528aae9c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:22.840027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be51ca2-9244-416b-a2a7-445528aae9c0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:22.883265 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 924/1850] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:22 2022] POST /identity/v3/projects => generated 387 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:13:22.889483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f469f71-a2dd-45d2-8b89-a06b2a009e3e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:22.900328 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f469f71-a2dd-45d2-8b89-a06b2a009e3e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:22.945169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:22.972199 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f57407-ed07-4190-8f69-fdc9828f4aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:22.972929 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f57407-ed07-4190-8f69-fdc9828f4aaa admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:22.973219 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f57407-ed07-4190-8f69-fdc9828f4aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:22.973497 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f57407-ed07-4190-8f69-fdc9828f4aaa admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:22.974155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f57407-ed07-4190-8f69-fdc9828f4aaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:22.977129 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f57407-ed07-4190-8f69-fdc9828f4aaa admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:23.002948 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 927/1851] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:22 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:13:23.004488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be51ca2-9244-416b-a2a7-445528aae9c0 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:23.016663 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be51ca2-9244-416b-a2a7-445528aae9c0 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:23.021630 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be67d8-7cb0-499c-9cd4-6f04470eed12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:23.022330 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be67d8-7cb0-499c-9cd4-6f04470eed12 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:23.022618 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be67d8-7cb0-499c-9cd4-6f04470eed12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:23.022918 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be67d8-7cb0-499c-9cd4-6f04470eed12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:23.023581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be67d8-7cb0-499c-9cd4-6f04470eed12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:23.025212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be67d8-7cb0-499c-9cd4-6f04470eed12 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:23.036565 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 925/1852] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:23.041622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f57407-ed07-4190-8f69-fdc9828f4aaa admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:23.052688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f57407-ed07-4190-8f69-fdc9828f4aaa admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:23.058753 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1328c-ae90-427e-83a0-a03ae289ff47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:23.059479 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1328c-ae90-427e-83a0-a03ae289ff47 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:23.059831 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1328c-ae90-427e-83a0-a03ae289ff47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:23.060291 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1328c-ae90-427e-83a0-a03ae289ff47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:23.061006 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1328c-ae90-427e-83a0-a03ae289ff47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:23.062672 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1328c-ae90-427e-83a0-a03ae289ff47 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:23.074899 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 928/1853] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:23.079472 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be67d8-7cb0-499c-9cd4-6f04470eed12 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:23.093618 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be67d8-7cb0-499c-9cd4-6f04470eed12 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:23.099409 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05ed8b-4a40-4745-b9be-7547e3be16f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:23.101062 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05ed8b-4a40-4745-b9be-7547e3be16f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:23.101348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05ed8b-4a40-4745-b9be-7547e3be16f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:23.102022 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05ed8b-4a40-4745-b9be-7547e3be16f7 admin admin] PATH_INFO: `/v3/projects/8e7f919406244681a5d03a2251e8437f/users/76098dcedddd4ae6812c7be4804fc7cb/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:23.118466 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05ed8b-4a40-4745-b9be-7547e3be16f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e7f919406244681a5d03a2251e8437f, user_id=76098dcedddd4ae6812c7be4804fc7cb, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:23.121410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05ed8b-4a40-4745-b9be-7547e3be16f7 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:23.137210 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 926/1854] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:23 2022] PUT /identity/v3/projects/8e7f919406244681a5d03a2251e8437f/users/76098dcedddd4ae6812c7be4804fc7cb/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:23.147640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514daae-e167-4f48-8db6-0faae4640b5f None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:23.147931 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514daae-e167-4f48-8db6-0faae4640b5f None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:23.148209 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514daae-e167-4f48-8db6-0faae4640b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:23.177879 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-8514daae-e167-4f48-8db6-0faae4640b5f None None] MFA Rules not processed for user `76098dcedddd4ae6812c7be4804fc7cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:23.179562 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8514daae-e167-4f48-8db6-0faae4640b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:23.251257 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 929/1855] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:23.255684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05ed8b-4a40-4745-b9be-7547e3be16f7 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:23.322217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05ed8b-4a40-4745-b9be-7547e3be16f7 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:23.326731 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfc032-3992-4baf-949e-9f0dd7fff603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:23.327371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfc032-3992-4baf-949e-9f0dd7fff603 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:23.327674 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfc032-3992-4baf-949e-9f0dd7fff603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:23.327947 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfc032-3992-4baf-949e-9f0dd7fff603 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:23.328583 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfc032-3992-4baf-949e-9f0dd7fff603 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:23.330486 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfc032-3992-4baf-949e-9f0dd7fff603 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:23.356628 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 927/1856] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:23 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:23.360635 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514daae-e167-4f48-8db6-0faae4640b5f None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:23.371033 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514daae-e167-4f48-8db6-0faae4640b5f None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:23.429709 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b35e6-7331-499a-92f2-31ad829ebe7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:23.430322 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b35e6-7331-499a-92f2-31ad829ebe7e admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:23.430601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b35e6-7331-499a-92f2-31ad829ebe7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:23.430898 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b35e6-7331-499a-92f2-31ad829ebe7e admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:23.431539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b35e6-7331-499a-92f2-31ad829ebe7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:23.433436 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b35e6-7331-499a-92f2-31ad829ebe7e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:23.456985 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 930/1857] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:23 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:23.458520 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfc032-3992-4baf-949e-9f0dd7fff603 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:23.469475 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfc032-3992-4baf-949e-9f0dd7fff603 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:23.474225 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dd14a-cadd-47d0-a4fd-fdb932eaa28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:23.478769 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dd14a-cadd-47d0-a4fd-fdb932eaa28e admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:23.479104 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dd14a-cadd-47d0-a4fd-fdb932eaa28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:23.479395 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dd14a-cadd-47d0-a4fd-fdb932eaa28e admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:23.480130 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dd14a-cadd-47d0-a4fd-fdb932eaa28e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:23.481811 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dd14a-cadd-47d0-a4fd-fdb932eaa28e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:23.494771 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 928/1858] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:23.498579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b35e6-7331-499a-92f2-31ad829ebe7e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:23.510723 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b35e6-7331-499a-92f2-31ad829ebe7e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:23.518057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd9cfd-646f-4fc8-946c-3707398153e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:23.518743 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd9cfd-646f-4fc8-946c-3707398153e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:23.519055 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd9cfd-646f-4fc8-946c-3707398153e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:23.519330 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd9cfd-646f-4fc8-946c-3707398153e4 admin admin] PATH_INFO: `/v3/projects/51356fd266aa42a8a635f6b9b7760491/users/084cca7db164477b98a15d4331399dc5/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:23.538212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd9cfd-646f-4fc8-946c-3707398153e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51356fd266aa42a8a635f6b9b7760491, user_id=084cca7db164477b98a15d4331399dc5, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:23.541103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd9cfd-646f-4fc8-946c-3707398153e4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:23.556288 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 931/1859] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:23 2022] PUT /identity/v3/projects/51356fd266aa42a8a635f6b9b7760491/users/084cca7db164477b98a15d4331399dc5/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:23.560358 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dd14a-cadd-47d0-a4fd-fdb932eaa28e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:23.571927 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dd14a-cadd-47d0-a4fd-fdb932eaa28e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:23.634129 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d5702-8e9d-4134-a0d4-46480feeaf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:23.634756 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d5702-8e9d-4134-a0d4-46480feeaf7b admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:23.635065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d5702-8e9d-4134-a0d4-46480feeaf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:23.635341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d5702-8e9d-4134-a0d4-46480feeaf7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:23.635988 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d5702-8e9d-4134-a0d4-46480feeaf7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:23.637581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d5702-8e9d-4134-a0d4-46480feeaf7b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:23.648824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 929/1860] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:23.653342 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd9cfd-646f-4fc8-946c-3707398153e4 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:23.664487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd9cfd-646f-4fc8-946c-3707398153e4 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:23.669467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265abd51-47c9-4669-8f72-4146085e22e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:23.670090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265abd51-47c9-4669-8f72-4146085e22e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:23.670417 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265abd51-47c9-4669-8f72-4146085e22e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:23.670715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265abd51-47c9-4669-8f72-4146085e22e2 admin admin] PATH_INFO: `/v3/projects/51356fd266aa42a8a635f6b9b7760491/users/084cca7db164477b98a15d4331399dc5/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:23.673034 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265abd51-47c9-4669-8f72-4146085e22e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51356fd266aa42a8a635f6b9b7760491, user_id=084cca7db164477b98a15d4331399dc5, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:23.675794 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265abd51-47c9-4669-8f72-4146085e22e2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:23.687889 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 932/1861] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:23 2022] PUT /identity/v3/projects/51356fd266aa42a8a635f6b9b7760491/users/084cca7db164477b98a15d4331399dc5/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:23.700538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be96dc-fcca-4b46-bc3f-a8fe404ce4df None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:23.701047 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be96dc-fcca-4b46-bc3f-a8fe404ce4df None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:23.701157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be96dc-fcca-4b46-bc3f-a8fe404ce4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:23.715445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:23.734700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-79be96dc-fcca-4b46-bc3f-a8fe404ce4df None None] MFA Rules not processed for user `084cca7db164477b98a15d4331399dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:23.736117 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-79be96dc-fcca-4b46-bc3f-a8fe404ce4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:23.821462 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 930/1862] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:23.834112 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265abd51-47c9-4669-8f72-4146085e22e2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:23.835953 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-265abd51-47c9-4669-8f72-4146085e22e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:23.841504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:23.899479 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265abd51-47c9-4669-8f72-4146085e22e2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:23.904056 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66bcb3c-2be8-4a51-bdc1-fafd96cfa5ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:23.904706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bcb3c-2be8-4a51-bdc1-fafd96cfa5ad service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:23.904989 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bcb3c-2be8-4a51-bdc1-fafd96cfa5ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:23.905301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66bcb3c-2be8-4a51-bdc1-fafd96cfa5ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:23.915749 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66bcb3c-2be8-4a51-bdc1-fafd96cfa5ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:23.917184 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66bcb3c-2be8-4a51-bdc1-fafd96cfa5ad service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:23.927118 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 933/1863] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:23 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:23.997429 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be96dc-fcca-4b46-bc3f-a8fe404ce4df None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:24.061060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be96dc-fcca-4b46-bc3f-a8fe404ce4df None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:24.065546 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45651d-e84d-4ca9-9264-9d39b016ba67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:24.066169 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45651d-e84d-4ca9-9264-9d39b016ba67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:24.066451 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45651d-e84d-4ca9-9264-9d39b016ba67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:24.066719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45651d-e84d-4ca9-9264-9d39b016ba67 admin admin] PATH_INFO: `/v3/users/76098dcedddd4ae6812c7be4804fc7cb` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:24.067860 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45651d-e84d-4ca9-9264-9d39b016ba67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76098dcedddd4ae6812c7be4804fc7cb)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:24.069794 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45651d-e84d-4ca9-9264-9d39b016ba67 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:24.114090 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ff45651d-e84d-4ca9-9264-9d39b016ba67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76098dcedddd4ae6812c7be4804fc7cb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:24.120859 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ff45651d-e84d-4ca9-9264-9d39b016ba67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76098dcedddd4ae6812c7be4804fc7cb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:24.121520 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ff45651d-e84d-4ca9-9264-9d39b016ba67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76098dcedddd4ae6812c7be4804fc7cb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:24.128326 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ff45651d-e84d-4ca9-9264-9d39b016ba67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76098dcedddd4ae6812c7be4804fc7cb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:24.140711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ff45651d-e84d-4ca9-9264-9d39b016ba67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76098dcedddd4ae6812c7be4804fc7cb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:24.147012 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 931/1864] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:23 2022] DELETE /identity/v3/users/76098dcedddd4ae6812c7be4804fc7cb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:24.151874 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3f9c5-0bec-4389-9dd0-e665a09a6eaa None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:24.152160 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3f9c5-0bec-4389-9dd0-e665a09a6eaa None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:24.152432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3f9c5-0bec-4389-9dd0-e665a09a6eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:24.171846 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-bfb3f9c5-0bec-4389-9dd0-e665a09a6eaa None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:24.173028 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-bfb3f9c5-0bec-4389-9dd0-e665a09a6eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:24.230234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 934/1865] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:24.239990 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45651d-e84d-4ca9-9264-9d39b016ba67 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:24.242112 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ff45651d-e84d-4ca9-9264-9d39b016ba67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:24.255557 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:24.301819 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45651d-e84d-4ca9-9264-9d39b016ba67 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:24.305970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e589f1-6f7c-40a0-af65-06f2d35b6fbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:24.306578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e589f1-6f7c-40a0-af65-06f2d35b6fbe service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:24.306855 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e589f1-6f7c-40a0-af65-06f2d35b6fbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:24.307155 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e589f1-6f7c-40a0-af65-06f2d35b6fbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:24.316300 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e589f1-6f7c-40a0-af65-06f2d35b6fbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:24.317730 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e589f1-6f7c-40a0-af65-06f2d35b6fbe service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:24.328052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 932/1866] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:24.603684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3f9c5-0bec-4389-9dd0-e665a09a6eaa None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:24.605180 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-bfb3f9c5-0bec-4389-9dd0-e665a09a6eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:24.618907 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3f9c5-0bec-4389-9dd0-e665a09a6eaa None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:24.623432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:24.624060 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:24.624341 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:24.624611 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c admin admin] PATH_INFO: `/v3/projects/8e7f919406244681a5d03a2251e8437f` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:24.625757 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e7f919406244681a5d03a2251e8437f)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:24.627673 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:24.678854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e7f919406244681a5d03a2251e8437f', 'request_id': 'req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:24.684540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e7f919406244681a5d03a2251e8437f', 'request_id': 'req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:24.690955 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e7f919406244681a5d03a2251e8437f', 'request_id': 'req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:24.704701 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c admin admin] The token cache is being invalidate because project 8e7f919406244681a5d03a2251e8437f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:24.704988 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:24.705615 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:24.707173 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 935/1867] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:24 2022] DELETE /identity/v3/projects/8e7f919406244681a5d03a2251e8437f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:24.711276 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e589f1-6f7c-40a0-af65-06f2d35b6fbe service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:24.721412 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e589f1-6f7c-40a0-af65-06f2d35b6fbe service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:24.780178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72d34e-dbc4-43f8-893d-4f31c1719347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:24.780865 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72d34e-dbc4-43f8-893d-4f31c1719347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:24.781149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72d34e-dbc4-43f8-893d-4f31c1719347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:24.781420 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72d34e-dbc4-43f8-893d-4f31c1719347 admin admin] PATH_INFO: `/v3/users/084cca7db164477b98a15d4331399dc5` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:24.782617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72d34e-dbc4-43f8-893d-4f31c1719347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=084cca7db164477b98a15d4331399dc5)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:24.784608 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72d34e-dbc4-43f8-893d-4f31c1719347 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:24.832185 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5e72d34e-dbc4-43f8-893d-4f31c1719347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '084cca7db164477b98a15d4331399dc5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:24.845147 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5e72d34e-dbc4-43f8-893d-4f31c1719347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '084cca7db164477b98a15d4331399dc5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:24.846017 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5e72d34e-dbc4-43f8-893d-4f31c1719347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '084cca7db164477b98a15d4331399dc5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:24.856216 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5e72d34e-dbc4-43f8-893d-4f31c1719347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '084cca7db164477b98a15d4331399dc5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:24.872205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5e72d34e-dbc4-43f8-893d-4f31c1719347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '084cca7db164477b98a15d4331399dc5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:24.878448 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 933/1868] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:24 2022] DELETE /identity/v3/users/084cca7db164477b98a15d4331399dc5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:25.519643 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:25.519643 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:25.572319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb363a8-9a39-4ba9-8132-cd710c3bed6c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:25.622774 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:25.668303 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f4ee1-7e94-4850-a24d-e1aae72cce3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:25.669240 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f4ee1-7e94-4850-a24d-e1aae72cce3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:25.669785 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f4ee1-7e94-4850-a24d-e1aae72cce3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:25.670264 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f4ee1-7e94-4850-a24d-e1aae72cce3a admin admin] PATH_INFO: `/v3/projects/51356fd266aa42a8a635f6b9b7760491` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:25.673561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f4ee1-7e94-4850-a24d-e1aae72cce3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=51356fd266aa42a8a635f6b9b7760491)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:25.677007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f4ee1-7e94-4850-a24d-e1aae72cce3a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:25.746894 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5e5f4ee1-7e94-4850-a24d-e1aae72cce3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51356fd266aa42a8a635f6b9b7760491', 'request_id': 'req-5e5f4ee1-7e94-4850-a24d-e1aae72cce3a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:25.752053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5e5f4ee1-7e94-4850-a24d-e1aae72cce3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51356fd266aa42a8a635f6b9b7760491', 'request_id': 'req-5e5f4ee1-7e94-4850-a24d-e1aae72cce3a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:25.764931 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5e5f4ee1-7e94-4850-a24d-e1aae72cce3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51356fd266aa42a8a635f6b9b7760491', 'request_id': 'req-5e5f4ee1-7e94-4850-a24d-e1aae72cce3a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:25.787434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5e5f4ee1-7e94-4850-a24d-e1aae72cce3a admin admin] The token cache is being invalidate because project 51356fd266aa42a8a635f6b9b7760491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:25.787744 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5e5f4ee1-7e94-4850-a24d-e1aae72cce3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:25.788603 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5e5f4ee1-7e94-4850-a24d-e1aae72cce3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:25.790737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 936/1869] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:25 2022] DELETE /identity/v3/projects/51356fd266aa42a8a635f6b9b7760491 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:25.798472 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04096e0-e79e-47df-9247-b310d2db0178 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:25.798966 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04096e0-e79e-47df-9247-b310d2db0178 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:25.799085 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04096e0-e79e-47df-9247-b310d2db0178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:25.831161 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-e04096e0-e79e-47df-9247-b310d2db0178 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:25.833201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e04096e0-e79e-47df-9247-b310d2db0178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:25.883220 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:25.965041 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 934/1870] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:25.972329 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691dba29-c21e-4ed8-bcfa-2bc657cda937 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:25.972614 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691dba29-c21e-4ed8-bcfa-2bc657cda937 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:25.972915 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691dba29-c21e-4ed8-bcfa-2bc657cda937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:26.009286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-691dba29-c21e-4ed8-bcfa-2bc657cda937 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:26.010499 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-691dba29-c21e-4ed8-bcfa-2bc657cda937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:26.023118 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 937/1871] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:26.024470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04096e0-e79e-47df-9247-b310d2db0178 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:26.038500 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04096e0-e79e-47df-9247-b310d2db0178 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:26.044870 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8851c-7c08-44bb-a7f2-1578f91ef8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:26.045687 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8851c-7c08-44bb-a7f2-1578f91ef8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:26.046055 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8851c-7c08-44bb-a7f2-1578f91ef8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:26.046523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8851c-7c08-44bb-a7f2-1578f91ef8b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:26.047316 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae8851c-7c08-44bb-a7f2-1578f91ef8b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:26.047766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae8851c-7c08-44bb-a7f2-1578f91ef8b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:26.049973 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae8851c-7c08-44bb-a7f2-1578f91ef8b8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:26.076185 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 935/1872] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:13:26.078429 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691dba29-c21e-4ed8-bcfa-2bc657cda937 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:26.091769 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691dba29-c21e-4ed8-bcfa-2bc657cda937 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:26.102226 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947af56-fea9-48e7-be74-3fcf916fdddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:26.103517 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947af56-fea9-48e7-be74-3fcf916fdddd admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:26.106339 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947af56-fea9-48e7-be74-3fcf916fdddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:26.106640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947af56-fea9-48e7-be74-3fcf916fdddd admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:26.107355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947af56-fea9-48e7-be74-3fcf916fdddd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:26.120886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947af56-fea9-48e7-be74-3fcf916fdddd admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:26.174544 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 938/1873] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:26 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:26.179324 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8851c-7c08-44bb-a7f2-1578f91ef8b8 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:26.196565 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8851c-7c08-44bb-a7f2-1578f91ef8b8 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:26.268607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c42b1-97b3-4995-b7b2-75d5d1c9dd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:26.269213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c42b1-97b3-4995-b7b2-75d5d1c9dd50 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:26.269835 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c42b1-97b3-4995-b7b2-75d5d1c9dd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:26.269835 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c42b1-97b3-4995-b7b2-75d5d1c9dd50 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:26.270512 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c42b1-97b3-4995-b7b2-75d5d1c9dd50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:26.273634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c42b1-97b3-4995-b7b2-75d5d1c9dd50 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:26.306157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 936/1874] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:26 2022] POST /identity/v3/users => generated 291 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:26.306366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947af56-fea9-48e7-be74-3fcf916fdddd admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:26.317396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947af56-fea9-48e7-be74-3fcf916fdddd admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:26.323707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3ae4d-f895-4567-9d63-9f5bf3af04c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:26.324549 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3ae4d-f895-4567-9d63-9f5bf3af04c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:26.324724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3ae4d-f895-4567-9d63-9f5bf3af04c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:26.325146 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3ae4d-f895-4567-9d63-9f5bf3af04c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:26.325605 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d3ae4d-f895-4567-9d63-9f5bf3af04c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:26.327233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d3ae4d-f895-4567-9d63-9f5bf3af04c7 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:26.338990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 939/1875] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:13:26.343318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c42b1-97b3-4995-b7b2-75d5d1c9dd50 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:26.354420 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c42b1-97b3-4995-b7b2-75d5d1c9dd50 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:26.359167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5205a-04c6-4188-a1ae-8a7a85f95e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:26.359773 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5205a-04c6-4188-a1ae-8a7a85f95e3d admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:26.360046 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5205a-04c6-4188-a1ae-8a7a85f95e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:26.360355 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5205a-04c6-4188-a1ae-8a7a85f95e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:26.360997 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5205a-04c6-4188-a1ae-8a7a85f95e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:26.362599 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5205a-04c6-4188-a1ae-8a7a85f95e3d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:26.376211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 937/1876] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:26.380965 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3ae4d-f895-4567-9d63-9f5bf3af04c7 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:26.394226 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3ae4d-f895-4567-9d63-9f5bf3af04c7 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:26.400168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a035c8a-5e37-4d2b-8689-72dd6754abd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:26.401704 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a035c8a-5e37-4d2b-8689-72dd6754abd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:26.401704 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a035c8a-5e37-4d2b-8689-72dd6754abd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:26.401704 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a035c8a-5e37-4d2b-8689-72dd6754abd2 admin admin] PATH_INFO: `/v3/projects/cbff53930496413ea9c1567256166778/users/64869bbb37164b9693fb79bc2de6f270/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:26.423057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a035c8a-5e37-4d2b-8689-72dd6754abd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbff53930496413ea9c1567256166778, user_id=64869bbb37164b9693fb79bc2de6f270, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:26.426905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a035c8a-5e37-4d2b-8689-72dd6754abd2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:26.457199 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 940/1877] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:26 2022] PUT /identity/v3/projects/cbff53930496413ea9c1567256166778/users/64869bbb37164b9693fb79bc2de6f270/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:26.459259 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf4a70-6089-4f33-860a-e47bc525a363 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:26.459660 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf4a70-6089-4f33-860a-e47bc525a363 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:26.460223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf4a70-6089-4f33-860a-e47bc525a363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:26.506719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-aebf4a70-6089-4f33-860a-e47bc525a363 None None] MFA Rules not processed for user `64869bbb37164b9693fb79bc2de6f270`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:26.507901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-aebf4a70-6089-4f33-860a-e47bc525a363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:26.575185 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 938/1878] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:13:26.579718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a035c8a-5e37-4d2b-8689-72dd6754abd2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:26.644933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:26.651470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a035c8a-5e37-4d2b-8689-72dd6754abd2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:26.658511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2815e-1e6f-4a3d-bbac-a29a533486e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:26.659159 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2815e-1e6f-4a3d-bbac-a29a533486e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:26.659444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2815e-1e6f-4a3d-bbac-a29a533486e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:26.659722 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2815e-1e6f-4a3d-bbac-a29a533486e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:26.660432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2815e-1e6f-4a3d-bbac-a29a533486e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:26.663016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2815e-1e6f-4a3d-bbac-a29a533486e3 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:26.691115 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 941/1879] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:26 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:26.695231 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf4a70-6089-4f33-860a-e47bc525a363 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:26.706144 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf4a70-6089-4f33-860a-e47bc525a363 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:26.741377 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:26.782715 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb7c27-99a4-4346-8f37-bdfb198ae153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:26.783438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb7c27-99a4-4346-8f37-bdfb198ae153 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:26.783791 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb7c27-99a4-4346-8f37-bdfb198ae153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:26.784136 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb7c27-99a4-4346-8f37-bdfb198ae153 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:26.784857 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb7c27-99a4-4346-8f37-bdfb198ae153 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:26.786848 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb7c27-99a4-4346-8f37-bdfb198ae153 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:26.809657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 939/1880] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:26 2022] POST /identity/v3/users => generated 297 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:26.813288 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2815e-1e6f-4a3d-bbac-a29a533486e3 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:26.825021 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2815e-1e6f-4a3d-bbac-a29a533486e3 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:26.829777 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33f53c-7d9d-4368-905c-242e37628c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:26.830432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33f53c-7d9d-4368-905c-242e37628c18 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:26.830718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33f53c-7d9d-4368-905c-242e37628c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:26.831023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33f53c-7d9d-4368-905c-242e37628c18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:26.831668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33f53c-7d9d-4368-905c-242e37628c18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:26.833239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33f53c-7d9d-4368-905c-242e37628c18 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:26.851422 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 942/1881] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:26.855827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb7c27-99a4-4346-8f37-bdfb198ae153 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:26.866511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb7c27-99a4-4346-8f37-bdfb198ae153 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:26.871603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c987f1-fa7a-46fb-9840-ad419c5352b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:26.872353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c987f1-fa7a-46fb-9840-ad419c5352b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:26.872566 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c987f1-fa7a-46fb-9840-ad419c5352b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:26.872844 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c987f1-fa7a-46fb-9840-ad419c5352b1 admin admin] PATH_INFO: `/v3/projects/ea35867fab004f47aea6feb8a91163f6/users/838da852fa7044469929551e4a8fd386/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:26.890355 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c987f1-fa7a-46fb-9840-ad419c5352b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea35867fab004f47aea6feb8a91163f6, user_id=838da852fa7044469929551e4a8fd386, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:26.893288 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c987f1-fa7a-46fb-9840-ad419c5352b1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:26.909261 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 940/1882] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:26 2022] PUT /identity/v3/projects/ea35867fab004f47aea6feb8a91163f6/users/838da852fa7044469929551e4a8fd386/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:26.920758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33f53c-7d9d-4368-905c-242e37628c18 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:26.932882 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33f53c-7d9d-4368-905c-242e37628c18 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:27.001961 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a365fb-db99-4d62-8b76-bf9e950d4285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:27.002527 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a365fb-db99-4d62-8b76-bf9e950d4285 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:27.002853 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a365fb-db99-4d62-8b76-bf9e950d4285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:27.003110 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a365fb-db99-4d62-8b76-bf9e950d4285 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:27.003766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a365fb-db99-4d62-8b76-bf9e950d4285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:27.006944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a365fb-db99-4d62-8b76-bf9e950d4285 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:27.026086 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 943/1883] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:27.031079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c987f1-fa7a-46fb-9840-ad419c5352b1 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:27.048852 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c987f1-fa7a-46fb-9840-ad419c5352b1 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:27.054098 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa4f65-540f-4798-9c3a-a9c1192798e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:27.054982 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa4f65-540f-4798-9c3a-a9c1192798e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:27.055362 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa4f65-540f-4798-9c3a-a9c1192798e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:27.055710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa4f65-540f-4798-9c3a-a9c1192798e5 admin admin] PATH_INFO: `/v3/projects/ea35867fab004f47aea6feb8a91163f6/users/838da852fa7044469929551e4a8fd386/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:27.057823 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa4f65-540f-4798-9c3a-a9c1192798e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea35867fab004f47aea6feb8a91163f6, user_id=838da852fa7044469929551e4a8fd386, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:27.060744 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa4f65-540f-4798-9c3a-a9c1192798e5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:27.086913 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 941/1884] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:27 2022] PUT /identity/v3/projects/ea35867fab004f47aea6feb8a91163f6/users/838da852fa7044469929551e4a8fd386/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:27.098998 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0931a7-7e53-4591-9cb9-28561943ed03 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:27.099742 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0931a7-7e53-4591-9cb9-28561943ed03 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:27.100102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0931a7-7e53-4591-9cb9-28561943ed03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:27.101816 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb631e6-a1fa-4899-b018-7c7cc6549a78 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:27.102109 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb631e6-a1fa-4899-b018-7c7cc6549a78 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:27.102386 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb631e6-a1fa-4899-b018-7c7cc6549a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:27.143183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ce0931a7-7e53-4591-9cb9-28561943ed03 None None] MFA Rules not processed for user `c58fbb2d79954020aa1328ef6bf925e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:27.143183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ce0931a7-7e53-4591-9cb9-28561943ed03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:27.148349 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-fcb631e6-a1fa-4899-b018-7c7cc6549a78 None None] MFA Rules not processed for user `838da852fa7044469929551e4a8fd386`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:27.149951 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-fcb631e6-a1fa-4899-b018-7c7cc6549a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:27.237904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 944/1885] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:27.252625 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 942/1886] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:13:27.269668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0931a7-7e53-4591-9cb9-28561943ed03 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:27.272018 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ce0931a7-7e53-4591-9cb9-28561943ed03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:27.370551 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0931a7-7e53-4591-9cb9-28561943ed03 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:27.377586 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bebb6d6-a464-4c20-b17d-1857328c8168 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:27.378564 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebb6d6-a464-4c20-b17d-1857328c8168 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:27.378978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebb6d6-a464-4c20-b17d-1857328c8168 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:27.379332 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bebb6d6-a464-4c20-b17d-1857328c8168 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:27.400368 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bebb6d6-a464-4c20-b17d-1857328c8168 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:27.402556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bebb6d6-a464-4c20-b17d-1857328c8168 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:27.416542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 945/1887] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:27.893121 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb631e6-a1fa-4899-b018-7c7cc6549a78 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:27.908801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb631e6-a1fa-4899-b018-7c7cc6549a78 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:27.914700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c532e7-37ed-48e3-abdb-bd2132031c9c tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '838da852fa7044469929551e4a8fd386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ea35867fab004f47aea6feb8a91163f6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:27.915379 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c532e7-37ed-48e3-abdb-bd2132031c9c tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:27.915664 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c532e7-37ed-48e3-abdb-bd2132031c9c tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:27.915943 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c532e7-37ed-48e3-abdb-bd2132031c9c tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:27.916599 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c532e7-37ed-48e3-abdb-bd2132031c9c tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:27.916907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c532e7-37ed-48e3-abdb-bd2132031c9c tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:27.918539 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c532e7-37ed-48e3-abdb-bd2132031c9c tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:27.931756 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 943/1888] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:13:27.935871 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bebb6d6-a464-4c20-b17d-1857328c8168 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:27.949318 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bebb6d6-a464-4c20-b17d-1857328c8168 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:27.954072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5cf19-74ab-4836-89a5-3de8796cbe89 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '838da852fa7044469929551e4a8fd386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ea35867fab004f47aea6feb8a91163f6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:27.954908 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5cf19-74ab-4836-89a5-3de8796cbe89 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:27.955488 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5cf19-74ab-4836-89a5-3de8796cbe89 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:27.955791 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5cf19-74ab-4836-89a5-3de8796cbe89 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:27.957576 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5cf19-74ab-4836-89a5-3de8796cbe89 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:27.961137 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5cf19-74ab-4836-89a5-3de8796cbe89 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:27.975113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:27.995730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 946/1889] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:27 2022] POST /identity/v3/projects => generated 354 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:13:27.999843 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c532e7-37ed-48e3-abdb-bd2132031c9c tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:28.025714 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c532e7-37ed-48e3-abdb-bd2132031c9c tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:28.089308 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:28.111170 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12e9fa-cbc4-426b-8877-d6ee43459761 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '838da852fa7044469929551e4a8fd386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ea35867fab004f47aea6feb8a91163f6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:28.112075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12e9fa-cbc4-426b-8877-d6ee43459761 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:28.112741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12e9fa-cbc4-426b-8877-d6ee43459761 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:28.113028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12e9fa-cbc4-426b-8877-d6ee43459761 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:28.113699 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12e9fa-cbc4-426b-8877-d6ee43459761 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:28.114440 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12e9fa-cbc4-426b-8877-d6ee43459761 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:28.116633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12e9fa-cbc4-426b-8877-d6ee43459761 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:28.135138 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 944/1890] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:13:28.180061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5cf19-74ab-4836-89a5-3de8796cbe89 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:28.201617 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5cf19-74ab-4836-89a5-3de8796cbe89 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:28.207930 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26de26-81d7-4e5b-bf02-481be865e51a tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '838da852fa7044469929551e4a8fd386', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ea35867fab004f47aea6feb8a91163f6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:28.208229 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26de26-81d7-4e5b-bf02-481be865e51a tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:28.208539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26de26-81d7-4e5b-bf02-481be865e51a tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:28.209153 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26de26-81d7-4e5b-bf02-481be865e51a tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] PATH_INFO: `/v3/projects/4b75f74375184af99e6a914f5756a3a2` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:28.210657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26de26-81d7-4e5b-bf02-481be865e51a tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4b75f74375184af99e6a914f5756a3a2)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:28.218207 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26de26-81d7-4e5b-bf02-481be865e51a tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:28.273697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ef26de26-81d7-4e5b-bf02-481be865e51a tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b75f74375184af99e6a914f5756a3a2', 'request_id': 'req-ef26de26-81d7-4e5b-bf02-481be865e51a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:28.286742 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ef26de26-81d7-4e5b-bf02-481be865e51a tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b75f74375184af99e6a914f5756a3a2', 'request_id': 'req-ef26de26-81d7-4e5b-bf02-481be865e51a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:28.294367 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ef26de26-81d7-4e5b-bf02-481be865e51a tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b75f74375184af99e6a914f5756a3a2', 'request_id': 'req-ef26de26-81d7-4e5b-bf02-481be865e51a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:28.310686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ef26de26-81d7-4e5b-bf02-481be865e51a tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] The token cache is being invalidate because project 4b75f74375184af99e6a914f5756a3a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:28.311240 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ef26de26-81d7-4e5b-bf02-481be865e51a tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:28.312050 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ef26de26-81d7-4e5b-bf02-481be865e51a tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:28.313573 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 947/1891] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:28 2022] DELETE /identity/v3/projects/4b75f74375184af99e6a914f5756a3a2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:28.514721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12e9fa-cbc4-426b-8877-d6ee43459761 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:28.517011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ca12e9fa-cbc4-426b-8877-d6ee43459761 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:28.587582 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12e9fa-cbc4-426b-8877-d6ee43459761 tempest-VolumeQuotaClassesTest-1951232198 tempest-VolumeQuotaClassesTest-1951232198-project-admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:28.659204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05afb9fe-a933-4991-acfe-e4413bc2981d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:28.660084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afb9fe-a933-4991-acfe-e4413bc2981d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:28.660200 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afb9fe-a933-4991-acfe-e4413bc2981d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:28.660508 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afb9fe-a933-4991-acfe-e4413bc2981d admin admin] PATH_INFO: `/v3/users/64869bbb37164b9693fb79bc2de6f270` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:28.671269 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05afb9fe-a933-4991-acfe-e4413bc2981d admin admin] RBAC: Authorizing `identity:delete_user(user_id=64869bbb37164b9693fb79bc2de6f270)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:28.673522 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05afb9fe-a933-4991-acfe-e4413bc2981d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:28.731794 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-05afb9fe-a933-4991-acfe-e4413bc2981d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64869bbb37164b9693fb79bc2de6f270'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:28.739172 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-05afb9fe-a933-4991-acfe-e4413bc2981d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64869bbb37164b9693fb79bc2de6f270'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:28.739900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-05afb9fe-a933-4991-acfe-e4413bc2981d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64869bbb37164b9693fb79bc2de6f270'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:28.748366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-05afb9fe-a933-4991-acfe-e4413bc2981d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64869bbb37164b9693fb79bc2de6f270'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:28.762493 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-05afb9fe-a933-4991-acfe-e4413bc2981d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64869bbb37164b9693fb79bc2de6f270'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:28.770042 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 945/1892] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:28 2022] DELETE /identity/v3/users/64869bbb37164b9693fb79bc2de6f270 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:28.775649 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcc14e-763a-4237-886f-6cd1ed6625a2 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:28.779213 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcc14e-763a-4237-886f-6cd1ed6625a2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:28.779736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcc14e-763a-4237-886f-6cd1ed6625a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:28.804045 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-76fcc14e-763a-4237-886f-6cd1ed6625a2 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:28.805273 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-76fcc14e-763a-4237-886f-6cd1ed6625a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:28.873775 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 948/1893] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:28.884079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05afb9fe-a933-4991-acfe-e4413bc2981d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:28.890497 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-05afb9fe-a933-4991-acfe-e4413bc2981d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:28.908570 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:28.977929 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05afb9fe-a933-4991-acfe-e4413bc2981d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:28.981740 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a9847-d1e3-4623-a6a1-09069ad58c97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:28.982414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a9847-d1e3-4623-a6a1-09069ad58c97 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:28.982631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a9847-d1e3-4623-a6a1-09069ad58c97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:28.982925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a9847-d1e3-4623-a6a1-09069ad58c97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:28.993818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a9847-d1e3-4623-a6a1-09069ad58c97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:28.995797 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a9847-d1e3-4623-a6a1-09069ad58c97 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:29.008697 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 946/1894] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:29.295711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fcc14e-763a-4237-886f-6cd1ed6625a2 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:29.297141 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-76fcc14e-763a-4237-886f-6cd1ed6625a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:29.309629 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fcc14e-763a-4237-886f-6cd1ed6625a2 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:29.314189 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:29.314841 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:29.315210 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:29.315478 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8 admin admin] PATH_INFO: `/v3/projects/cbff53930496413ea9c1567256166778` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:29.316592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbff53930496413ea9c1567256166778)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:29.319104 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:29.366732 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbff53930496413ea9c1567256166778', 'request_id': 'req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:29.373798 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbff53930496413ea9c1567256166778', 'request_id': 'req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:29.380402 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbff53930496413ea9c1567256166778', 'request_id': 'req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:29.389336 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=83,keystone:INSERT=3,keystone:DELETE=14 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:29.396023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8 admin admin] The token cache is being invalidate because project cbff53930496413ea9c1567256166778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:29.396331 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:29.397706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:29.402377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 949/1895] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:29 2022] DELETE /identity/v3/projects/cbff53930496413ea9c1567256166778 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:29.406711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a9847-d1e3-4623-a6a1-09069ad58c97 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:29.417182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a9847-d1e3-4623-a6a1-09069ad58c97 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:29.496630 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31071667-3d16-4345-853d-3997365b708e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:29.497259 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31071667-3d16-4345-853d-3997365b708e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:29.497547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31071667-3d16-4345-853d-3997365b708e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:29.497818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31071667-3d16-4345-853d-3997365b708e admin admin] PATH_INFO: `/v3/users/838da852fa7044469929551e4a8fd386` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:29.499027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31071667-3d16-4345-853d-3997365b708e admin admin] RBAC: Authorizing `identity:delete_user(user_id=838da852fa7044469929551e4a8fd386)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:29.501026 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31071667-3d16-4345-853d-3997365b708e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:29.553770 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-31071667-3d16-4345-853d-3997365b708e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '838da852fa7044469929551e4a8fd386'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:29.561303 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-31071667-3d16-4345-853d-3997365b708e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '838da852fa7044469929551e4a8fd386'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:29.562013 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-31071667-3d16-4345-853d-3997365b708e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '838da852fa7044469929551e4a8fd386'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:29.570272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-31071667-3d16-4345-853d-3997365b708e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '838da852fa7044469929551e4a8fd386'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:29.588363 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-31071667-3d16-4345-853d-3997365b708e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '838da852fa7044469929551e4a8fd386'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:29.595278 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 947/1896] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:29 2022] DELETE /identity/v3/users/838da852fa7044469929551e4a8fd386 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:30.130896 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:30.133511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:30.156725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f18bd9-2966-45f7-98ba-1f1a3ccfebb8 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:30.224195 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4471798-b85f-4a28-ad9d-ed98d4921114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:30.225166 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4471798-b85f-4a28-ad9d-ed98d4921114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:30.225454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4471798-b85f-4a28-ad9d-ed98d4921114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:30.225737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4471798-b85f-4a28-ad9d-ed98d4921114 admin admin] PATH_INFO: `/v3/projects/ea35867fab004f47aea6feb8a91163f6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:30.226976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4471798-b85f-4a28-ad9d-ed98d4921114 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea35867fab004f47aea6feb8a91163f6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:30.229161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4471798-b85f-4a28-ad9d-ed98d4921114 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:30.299447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e4471798-b85f-4a28-ad9d-ed98d4921114 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea35867fab004f47aea6feb8a91163f6', 'request_id': 'req-e4471798-b85f-4a28-ad9d-ed98d4921114'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:30.311164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e4471798-b85f-4a28-ad9d-ed98d4921114 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea35867fab004f47aea6feb8a91163f6', 'request_id': 'req-e4471798-b85f-4a28-ad9d-ed98d4921114'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:30.319296 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e4471798-b85f-4a28-ad9d-ed98d4921114 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea35867fab004f47aea6feb8a91163f6', 'request_id': 'req-e4471798-b85f-4a28-ad9d-ed98d4921114'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:30.356362 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e4471798-b85f-4a28-ad9d-ed98d4921114 admin admin] The token cache is being invalidate because project ea35867fab004f47aea6feb8a91163f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:30.356803 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e4471798-b85f-4a28-ad9d-ed98d4921114 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:30.357671 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e4471798-b85f-4a28-ad9d-ed98d4921114 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:30.359313 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 950/1897] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:30 2022] DELETE /identity/v3/projects/ea35867fab004f47aea6feb8a91163f6 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:13:30.364309 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e4d11-dd10-44ee-94dc-3f6d065fa61a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:30.364777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e4d11-dd10-44ee-94dc-3f6d065fa61a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:30.364915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e4d11-dd10-44ee-94dc-3f6d065fa61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:30.392726 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b54e4d11-dd10-44ee-94dc-3f6d065fa61a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:30.394081 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b54e4d11-dd10-44ee-94dc-3f6d065fa61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:30.434492 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:30.479993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 948/1898] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:30.480751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d6b9c-f9f0-4383-b3c9-9cc50a496cd3 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:30.481070 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d6b9c-f9f0-4383-b3c9-9cc50a496cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:30.481407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d6b9c-f9f0-4383-b3c9-9cc50a496cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:30.507026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6c6d6b9c-f9f0-4383-b3c9-9cc50a496cd3 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:30.508237 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6c6d6b9c-f9f0-4383-b3c9-9cc50a496cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:30.519240 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 951/1899] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:13:30.524100 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e4d11-dd10-44ee-94dc-3f6d065fa61a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:30.534348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e4d11-dd10-44ee-94dc-3f6d065fa61a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:30.540623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b81bf9-f78f-44a9-8638-ae2690acf5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:30.541240 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b81bf9-f78f-44a9-8638-ae2690acf5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:30.541524 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b81bf9-f78f-44a9-8638-ae2690acf5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:30.541797 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b81bf9-f78f-44a9-8638-ae2690acf5f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:30.542448 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b81bf9-f78f-44a9-8638-ae2690acf5f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:30.542902 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b81bf9-f78f-44a9-8638-ae2690acf5f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:30.544421 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b81bf9-f78f-44a9-8638-ae2690acf5f1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:30.561216 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 949/1900] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:13:30.565304 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d6b9c-f9f0-4383-b3c9-9cc50a496cd3 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:30.574976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d6b9c-f9f0-4383-b3c9-9cc50a496cd3 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:30.580177 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f6786-c7a5-4b69-881c-3bab796df2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:30.580776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f6786-c7a5-4b69-881c-3bab796df2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:30.581089 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f6786-c7a5-4b69-881c-3bab796df2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:30.581360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f6786-c7a5-4b69-881c-3bab796df2e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:30.582409 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f6786-c7a5-4b69-881c-3bab796df2e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:30.585020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f6786-c7a5-4b69-881c-3bab796df2e5 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:30.620247 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 952/1901] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:30 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:30.624390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b81bf9-f78f-44a9-8638-ae2690acf5f1 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:30.637389 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b81bf9-f78f-44a9-8638-ae2690acf5f1 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:30.706475 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b396685-189b-489d-a9f7-9f7fc1e3e775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:30.707128 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b396685-189b-489d-a9f7-9f7fc1e3e775 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:30.707415 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b396685-189b-489d-a9f7-9f7fc1e3e775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:30.707693 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b396685-189b-489d-a9f7-9f7fc1e3e775 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:30.708349 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b396685-189b-489d-a9f7-9f7fc1e3e775 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:30.710433 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b396685-189b-489d-a9f7-9f7fc1e3e775 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:30.734858 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 950/1902] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:30 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:30.739863 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f6786-c7a5-4b69-881c-3bab796df2e5 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:30.752080 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f6786-c7a5-4b69-881c-3bab796df2e5 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:30.757560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe38de6-81c8-4663-aa64-0e5583a11c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:30.758172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe38de6-81c8-4663-aa64-0e5583a11c62 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:30.758513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe38de6-81c8-4663-aa64-0e5583a11c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:30.758794 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe38de6-81c8-4663-aa64-0e5583a11c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:30.759507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe38de6-81c8-4663-aa64-0e5583a11c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:30.761201 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe38de6-81c8-4663-aa64-0e5583a11c62 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:30.781003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 953/1903] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:30.782336 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b396685-189b-489d-a9f7-9f7fc1e3e775 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:30.797585 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b396685-189b-489d-a9f7-9f7fc1e3e775 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:30.804171 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b737e3d-bfe2-4d60-a2f6-259ebdc03d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:30.804783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b737e3d-bfe2-4d60-a2f6-259ebdc03d47 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:30.805056 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b737e3d-bfe2-4d60-a2f6-259ebdc03d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:30.805329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b737e3d-bfe2-4d60-a2f6-259ebdc03d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:30.805969 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b737e3d-bfe2-4d60-a2f6-259ebdc03d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:30.807605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b737e3d-bfe2-4d60-a2f6-259ebdc03d47 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:30.819162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 951/1904] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:30.823992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe38de6-81c8-4663-aa64-0e5583a11c62 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:30.834763 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe38de6-81c8-4663-aa64-0e5583a11c62 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:30.839743 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981df1d-f6a8-4fca-b2c2-5db96ac64083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:30.840407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981df1d-f6a8-4fca-b2c2-5db96ac64083 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:30.840736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981df1d-f6a8-4fca-b2c2-5db96ac64083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:30.840968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981df1d-f6a8-4fca-b2c2-5db96ac64083 admin admin] PATH_INFO: `/v3/projects/fe184d1bd78b46d998114c0ea46793d4/users/684c67b8959c43f5bde0632a02d0e630/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:30.858156 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981df1d-f6a8-4fca-b2c2-5db96ac64083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe184d1bd78b46d998114c0ea46793d4, user_id=684c67b8959c43f5bde0632a02d0e630, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:30.861351 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981df1d-f6a8-4fca-b2c2-5db96ac64083 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:30.875465 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 954/1905] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:30 2022] PUT /identity/v3/projects/fe184d1bd78b46d998114c0ea46793d4/users/684c67b8959c43f5bde0632a02d0e630/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:30.886779 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69316c9-d85f-41dd-99e1-53b58cbed021 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:30.887115 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69316c9-d85f-41dd-99e1-53b58cbed021 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:30.887576 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69316c9-d85f-41dd-99e1-53b58cbed021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:30.920232 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f69316c9-d85f-41dd-99e1-53b58cbed021 None None] MFA Rules not processed for user `684c67b8959c43f5bde0632a02d0e630`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:30.921510 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f69316c9-d85f-41dd-99e1-53b58cbed021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:31.009754 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 952/1906] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:13:31.014221 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981df1d-f6a8-4fca-b2c2-5db96ac64083 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:31.046262 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:31.090037 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981df1d-f6a8-4fca-b2c2-5db96ac64083 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:31.095181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800215e-3df1-4a51-857e-0f191ff1b36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:31.095799 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800215e-3df1-4a51-857e-0f191ff1b36e admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:31.096082 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800215e-3df1-4a51-857e-0f191ff1b36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:31.096354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800215e-3df1-4a51-857e-0f191ff1b36e admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:31.097805 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800215e-3df1-4a51-857e-0f191ff1b36e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:31.100976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800215e-3df1-4a51-857e-0f191ff1b36e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:31.132350 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 955/1907] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:31 2022] POST /identity/v3/projects => generated 391 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:13:31.139071 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69316c9-d85f-41dd-99e1-53b58cbed021 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:31.156127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69316c9-d85f-41dd-99e1-53b58cbed021 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:31.191284 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:31.248448 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f948e1-3b47-4de4-83b4-600fbd6b5f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:31.249160 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f948e1-3b47-4de4-83b4-600fbd6b5f60 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:31.249407 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f948e1-3b47-4de4-83b4-600fbd6b5f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:31.249698 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f948e1-3b47-4de4-83b4-600fbd6b5f60 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:31.250373 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f948e1-3b47-4de4-83b4-600fbd6b5f60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:31.252930 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f948e1-3b47-4de4-83b4-600fbd6b5f60 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:31.283039 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800215e-3df1-4a51-857e-0f191ff1b36e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:31.287952 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a800215e-3df1-4a51-857e-0f191ff1b36e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:31.314921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 953/1908] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:31 2022] POST /identity/v3/users => generated 302 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:31.316633 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800215e-3df1-4a51-857e-0f191ff1b36e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:31.317499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f948e1-3b47-4de4-83b4-600fbd6b5f60 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:31.331303 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f948e1-3b47-4de4-83b4-600fbd6b5f60 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:31.337553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044df05-7717-4f78-b078-8f8489ce00c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:31.338333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044df05-7717-4f78-b078-8f8489ce00c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:31.338704 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044df05-7717-4f78-b078-8f8489ce00c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:31.339103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044df05-7717-4f78-b078-8f8489ce00c4 admin admin] PATH_INFO: `/v3/users/15739afe0c9341de996c6f963b08e52a` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:31.340495 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044df05-7717-4f78-b078-8f8489ce00c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15739afe0c9341de996c6f963b08e52a)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:31.341348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bc6f4-bd50-47cf-8269-e8d4890e7c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:31.342194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bc6f4-bd50-47cf-8269-e8d4890e7c61 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:31.342670 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bc6f4-bd50-47cf-8269-e8d4890e7c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:31.342820 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044df05-7717-4f78-b078-8f8489ce00c4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:31.343457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bc6f4-bd50-47cf-8269-e8d4890e7c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:31.344317 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509bc6f4-bd50-47cf-8269-e8d4890e7c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:31.346275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509bc6f4-bd50-47cf-8269-e8d4890e7c61 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:31.359662 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 954/1909] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:31.363626 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bc6f4-bd50-47cf-8269-e8d4890e7c61 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:31.379079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bc6f4-bd50-47cf-8269-e8d4890e7c61 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:31.385156 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c0356-d621-4154-9b79-602ff77cdce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:31.386427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c0356-d621-4154-9b79-602ff77cdce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:31.386427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c0356-d621-4154-9b79-602ff77cdce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:31.386427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c0356-d621-4154-9b79-602ff77cdce9 admin admin] PATH_INFO: `/v3/projects/43c90118339543b8b7420fd319a8c9cc/users/a613fb99491f4e7b9c012c7422dff842/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:31.410490 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d044df05-7717-4f78-b078-8f8489ce00c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15739afe0c9341de996c6f963b08e52a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:31.410916 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c0356-d621-4154-9b79-602ff77cdce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c90118339543b8b7420fd319a8c9cc, user_id=a613fb99491f4e7b9c012c7422dff842, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:31.414572 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c0356-d621-4154-9b79-602ff77cdce9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:31.420233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d044df05-7717-4f78-b078-8f8489ce00c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15739afe0c9341de996c6f963b08e52a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:31.421232 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d044df05-7717-4f78-b078-8f8489ce00c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15739afe0c9341de996c6f963b08e52a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:31.430509 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 955/1910] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:31 2022] PUT /identity/v3/projects/43c90118339543b8b7420fd319a8c9cc/users/a613fb99491f4e7b9c012c7422dff842/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:13:31.432290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d044df05-7717-4f78-b078-8f8489ce00c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15739afe0c9341de996c6f963b08e52a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:31.435431 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c0356-d621-4154-9b79-602ff77cdce9 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:31.440291 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3d1c0356-d621-4154-9b79-602ff77cdce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:31.448552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d044df05-7717-4f78-b078-8f8489ce00c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15739afe0c9341de996c6f963b08e52a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:31.454921 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 956/1911] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:31 2022] DELETE /identity/v3/users/15739afe0c9341de996c6f963b08e52a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:13:31.459486 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099bf581-ad22-43f8-843e-ac6cd44383a0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:31.459790 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099bf581-ad22-43f8-843e-ac6cd44383a0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:31.460064 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099bf581-ad22-43f8-843e-ac6cd44383a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:31.482735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-099bf581-ad22-43f8-843e-ac6cd44383a0 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:31.486177 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-099bf581-ad22-43f8-843e-ac6cd44383a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:31.524284 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c0356-d621-4154-9b79-602ff77cdce9 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:31.554278 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 957/1912] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:13:31.565858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099bf581-ad22-43f8-843e-ac6cd44383a0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:31.567368 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-099bf581-ad22-43f8-843e-ac6cd44383a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:31.599547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f852179-3c38-4449-a3c9-a9dc5394bd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:31.600900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f852179-3c38-4449-a3c9-a9dc5394bd16 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:31.601184 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f852179-3c38-4449-a3c9-a9dc5394bd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:31.601455 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f852179-3c38-4449-a3c9-a9dc5394bd16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:31.602186 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f852179-3c38-4449-a3c9-a9dc5394bd16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:31.604051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f852179-3c38-4449-a3c9-a9dc5394bd16 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:31.606322 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:31.619342 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 956/1913] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:13:31.620299 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f852179-3c38-4449-a3c9-a9dc5394bd16 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:31.633910 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f852179-3c38-4449-a3c9-a9dc5394bd16 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:31.639314 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de441f29-790d-4a5d-a513-ca7f6a350b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:31.639979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de441f29-790d-4a5d-a513-ca7f6a350b57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:31.640254 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de441f29-790d-4a5d-a513-ca7f6a350b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:31.640522 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de441f29-790d-4a5d-a513-ca7f6a350b57 admin admin] PATH_INFO: `/v3/projects/43c90118339543b8b7420fd319a8c9cc/users/a613fb99491f4e7b9c012c7422dff842/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:31.642633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de441f29-790d-4a5d-a513-ca7f6a350b57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c90118339543b8b7420fd319a8c9cc, user_id=a613fb99491f4e7b9c012c7422dff842, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:31.645820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de441f29-790d-4a5d-a513-ca7f6a350b57 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:31.652626 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099bf581-ad22-43f8-843e-ac6cd44383a0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:31.659232 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 957/1914] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:31 2022] PUT /identity/v3/projects/43c90118339543b8b7420fd319a8c9cc/users/a613fb99491f4e7b9c012c7422dff842/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:13:31.670258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bf7de-739c-4c94-8dbc-1f9c8fdbe4b4 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:31.670659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bf7de-739c-4c94-8dbc-1f9c8fdbe4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:31.671005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2bf7de-739c-4c94-8dbc-1f9c8fdbe4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:31.703880 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8e2bf7de-739c-4c94-8dbc-1f9c8fdbe4b4 None None] MFA Rules not processed for user `a613fb99491f4e7b9c012c7422dff842`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:31.704580 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8e2bf7de-739c-4c94-8dbc-1f9c8fdbe4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:31.720476 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:31.724872 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2b61a-3d1b-4d3a-b46d-413076f0ee62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:31.725649 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2b61a-3d1b-4d3a-b46d-413076f0ee62 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:31.726001 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2b61a-3d1b-4d3a-b46d-413076f0ee62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:31.727698 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2b61a-3d1b-4d3a-b46d-413076f0ee62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:31.813378 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 958/1915] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:31 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:31.822395 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d2b61a-3d1b-4d3a-b46d-413076f0ee62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:31.824739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d2b61a-3d1b-4d3a-b46d-413076f0ee62 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:31.826436 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bf7de-739c-4c94-8dbc-1f9c8fdbe4b4 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:31.828234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8e2bf7de-739c-4c94-8dbc-1f9c8fdbe4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:31.857999 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 958/1916] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:13:31.917925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2bf7de-739c-4c94-8dbc-1f9c8fdbe4b4 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:31.923166 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a23df7-bd5d-465e-9bce-0db22674c11e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:31.924082 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a23df7-bd5d-465e-9bce-0db22674c11e service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:31.925233 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a23df7-bd5d-465e-9bce-0db22674c11e service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:31.925233 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a23df7-bd5d-465e-9bce-0db22674c11e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:31.936064 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a23df7-bd5d-465e-9bce-0db22674c11e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:31.937532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a23df7-bd5d-465e-9bce-0db22674c11e service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:31.948223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 959/1917] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:31 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:32.000217 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2b61a-3d1b-4d3a-b46d-413076f0ee62 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:32.001634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a6d2b61a-3d1b-4d3a-b46d-413076f0ee62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:32.014078 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2b61a-3d1b-4d3a-b46d-413076f0ee62 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:32.018089 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d971a81-ec30-4e5c-b381-8114b97668c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:32.018722 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d971a81-ec30-4e5c-b381-8114b97668c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:32.019057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d971a81-ec30-4e5c-b381-8114b97668c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:32.019318 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d971a81-ec30-4e5c-b381-8114b97668c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:32.022996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8d971a81-ec30-4e5c-b381-8114b97668c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:32.091046 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d971a81-ec30-4e5c-b381-8114b97668c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:32.092653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d971a81-ec30-4e5c-b381-8114b97668c5 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:32.103011 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 959/1918] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:31 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:32.164949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a23df7-bd5d-465e-9bce-0db22674c11e service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:32.166427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-04a23df7-bd5d-465e-9bce-0db22674c11e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:32.181923 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a23df7-bd5d-465e-9bce-0db22674c11e service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:32.186796 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853b226-9df4-4c1a-8e80-3fcb73a63848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:32.187369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853b226-9df4-4c1a-8e80-3fcb73a63848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:32.187673 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853b226-9df4-4c1a-8e80-3fcb73a63848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:32.187943 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853b226-9df4-4c1a-8e80-3fcb73a63848 admin admin] PATH_INFO: `/v3/projects/4b832feb5f7644a9b93ce9e99af3ef11` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:32.189070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853b226-9df4-4c1a-8e80-3fcb73a63848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b832feb5f7644a9b93ce9e99af3ef11)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:32.191142 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853b226-9df4-4c1a-8e80-3fcb73a63848 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:32.243128 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-8853b226-9df4-4c1a-8e80-3fcb73a63848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b832feb5f7644a9b93ce9e99af3ef11', 'request_id': 'req-8853b226-9df4-4c1a-8e80-3fcb73a63848'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:32.250294 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-8853b226-9df4-4c1a-8e80-3fcb73a63848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b832feb5f7644a9b93ce9e99af3ef11', 'request_id': 'req-8853b226-9df4-4c1a-8e80-3fcb73a63848'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:32.257591 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-8853b226-9df4-4c1a-8e80-3fcb73a63848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b832feb5f7644a9b93ce9e99af3ef11', 'request_id': 'req-8853b226-9df4-4c1a-8e80-3fcb73a63848'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:32.269098 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-8853b226-9df4-4c1a-8e80-3fcb73a63848 admin admin] The token cache is being invalidate because project 4b832feb5f7644a9b93ce9e99af3ef11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:32.269391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-8853b226-9df4-4c1a-8e80-3fcb73a63848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:32.270011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-8853b226-9df4-4c1a-8e80-3fcb73a63848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:32.271601 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 960/1919] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:32 2022] DELETE /identity/v3/projects/4b832feb5f7644a9b93ce9e99af3ef11 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:32.276668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0916a-e53a-43a3-8f1f-9ef4d889ca60 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:32.276975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0916a-e53a-43a3-8f1f-9ef4d889ca60 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:32.277252 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0916a-e53a-43a3-8f1f-9ef4d889ca60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:32.301120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3be0916a-e53a-43a3-8f1f-9ef4d889ca60 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:32.302431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3be0916a-e53a-43a3-8f1f-9ef4d889ca60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:32.324187 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:32.375705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 960/1920] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:32.384684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9767a-b114-48c6-9fe9-2b2cb61cdf94 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:32.385144 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9767a-b114-48c6-9fe9-2b2cb61cdf94 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:32.385314 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9767a-b114-48c6-9fe9-2b2cb61cdf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:32.408490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-bce9767a-b114-48c6-9fe9-2b2cb61cdf94 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:32.409799 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bce9767a-b114-48c6-9fe9-2b2cb61cdf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:32.429034 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 961/1921] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:32.429965 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0916a-e53a-43a3-8f1f-9ef4d889ca60 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:32.442002 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0916a-e53a-43a3-8f1f-9ef4d889ca60 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:32.462908 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671d054-b31a-46c7-ba33-a73877a75305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:32.462908 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671d054-b31a-46c7-ba33-a73877a75305 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:32.462908 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671d054-b31a-46c7-ba33-a73877a75305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:32.462908 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671d054-b31a-46c7-ba33-a73877a75305 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:32.463591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671d054-b31a-46c7-ba33-a73877a75305 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:32.463591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671d054-b31a-46c7-ba33-a73877a75305 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:32.463591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2671d054-b31a-46c7-ba33-a73877a75305 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:32.469909 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 961/1922] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:13:32.473369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9767a-b114-48c6-9fe9-2b2cb61cdf94 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:32.484380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9767a-b114-48c6-9fe9-2b2cb61cdf94 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:32.496527 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337afa2-909a-482a-b995-f2f8b398b05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:32.497074 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337afa2-909a-482a-b995-f2f8b398b05f admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:32.497353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337afa2-909a-482a-b995-f2f8b398b05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:32.497627 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337afa2-909a-482a-b995-f2f8b398b05f admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:32.498280 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337afa2-909a-482a-b995-f2f8b398b05f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:32.500240 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337afa2-909a-482a-b995-f2f8b398b05f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:32.524561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:32.533377 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 962/1923] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:32 2022] POST /identity/v3/projects => generated 391 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:13:32.537609 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671d054-b31a-46c7-ba33-a73877a75305 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:32.548472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2671d054-b31a-46c7-ba33-a73877a75305 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:32.608207 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61ff92-8411-46ac-ba4a-20e16c8e1cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:32.608801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61ff92-8411-46ac-ba4a-20e16c8e1cf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:32.609116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61ff92-8411-46ac-ba4a-20e16c8e1cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:32.609394 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61ff92-8411-46ac-ba4a-20e16c8e1cf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:32.610040 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61ff92-8411-46ac-ba4a-20e16c8e1cf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:32.612042 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61ff92-8411-46ac-ba4a-20e16c8e1cf3 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:32.636295 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 962/1924] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:32 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:32.637472 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337afa2-909a-482a-b995-f2f8b398b05f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:32.648708 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337afa2-909a-482a-b995-f2f8b398b05f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:32.653514 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125c6b6-b9ec-45c8-ae7e-5a240418142f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:32.654110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125c6b6-b9ec-45c8-ae7e-5a240418142f admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:32.654484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125c6b6-b9ec-45c8-ae7e-5a240418142f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:32.654910 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125c6b6-b9ec-45c8-ae7e-5a240418142f admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:32.655961 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125c6b6-b9ec-45c8-ae7e-5a240418142f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:32.658294 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a125c6b6-b9ec-45c8-ae7e-5a240418142f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:32.669478 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 963/1925] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:13:32.673490 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61ff92-8411-46ac-ba4a-20e16c8e1cf3 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:32.684355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61ff92-8411-46ac-ba4a-20e16c8e1cf3 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:32.689433 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56091f8d-5635-4e9b-b398-9941a066a12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:32.690028 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56091f8d-5635-4e9b-b398-9941a066a12c admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:32.690306 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56091f8d-5635-4e9b-b398-9941a066a12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:32.690577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56091f8d-5635-4e9b-b398-9941a066a12c admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:32.691280 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56091f8d-5635-4e9b-b398-9941a066a12c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:32.692858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56091f8d-5635-4e9b-b398-9941a066a12c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:32.706234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 963/1926] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:32.710380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125c6b6-b9ec-45c8-ae7e-5a240418142f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:32.720543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125c6b6-b9ec-45c8-ae7e-5a240418142f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:32.726278 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d07cb3-af2d-421c-b148-dc973be9a47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:32.726992 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d07cb3-af2d-421c-b148-dc973be9a47e admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:32.727279 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d07cb3-af2d-421c-b148-dc973be9a47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:32.727555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d07cb3-af2d-421c-b148-dc973be9a47e admin admin] PATH_INFO: `/v3/projects/2c415e743af644fc834efc9ad512898a/users/7218a9eec3d8414d86bd505bd2d06425/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:32.743275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d07cb3-af2d-421c-b148-dc973be9a47e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c415e743af644fc834efc9ad512898a, user_id=7218a9eec3d8414d86bd505bd2d06425, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:32.745965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d07cb3-af2d-421c-b148-dc973be9a47e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:32.759850 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 964/1927] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:32 2022] PUT /identity/v3/projects/2c415e743af644fc834efc9ad512898a/users/7218a9eec3d8414d86bd505bd2d06425/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:32.770444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e3aad-8c8a-4fff-a8c9-100c2e0c3a9c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:32.770737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e3aad-8c8a-4fff-a8c9-100c2e0c3a9c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:32.771061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e3aad-8c8a-4fff-a8c9-100c2e0c3a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:32.799935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-954e3aad-8c8a-4fff-a8c9-100c2e0c3a9c None None] MFA Rules not processed for user `7218a9eec3d8414d86bd505bd2d06425`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:32.801155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-954e3aad-8c8a-4fff-a8c9-100c2e0c3a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:32.866776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 964/1928] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:32.878547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d07cb3-af2d-421c-b148-dc973be9a47e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:32.880306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f3d07cb3-af2d-421c-b148-dc973be9a47e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:32.942984 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d07cb3-af2d-421c-b148-dc973be9a47e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:32.947401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c020932-6871-49b4-9afb-5e73b9c53a83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:32.948014 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c020932-6871-49b4-9afb-5e73b9c53a83 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:32.948295 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c020932-6871-49b4-9afb-5e73b9c53a83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:32.948567 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c020932-6871-49b4-9afb-5e73b9c53a83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:32.957897 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c020932-6871-49b4-9afb-5e73b9c53a83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:32.959893 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c020932-6871-49b4-9afb-5e73b9c53a83 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:32.970255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 965/1929] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:13:35.648537 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e3aad-8c8a-4fff-a8c9-100c2e0c3a9c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:35.650064 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-954e3aad-8c8a-4fff-a8c9-100c2e0c3a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:35.676474 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:35.716294 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e3aad-8c8a-4fff-a8c9-100c2e0c3a9c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:35.720858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28a6ba-db97-4d40-8445-2076ca5580ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:35.721493 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28a6ba-db97-4d40-8445-2076ca5580ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:35.721834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28a6ba-db97-4d40-8445-2076ca5580ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:35.722048 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28a6ba-db97-4d40-8445-2076ca5580ff admin admin] PATH_INFO: `/v3/users/684c67b8959c43f5bde0632a02d0e630` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:35.723188 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c28a6ba-db97-4d40-8445-2076ca5580ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=684c67b8959c43f5bde0632a02d0e630)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:35.725440 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c28a6ba-db97-4d40-8445-2076ca5580ff admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:35.773559 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2c28a6ba-db97-4d40-8445-2076ca5580ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '684c67b8959c43f5bde0632a02d0e630'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:35.780604 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2c28a6ba-db97-4d40-8445-2076ca5580ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '684c67b8959c43f5bde0632a02d0e630'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:35.781255 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2c28a6ba-db97-4d40-8445-2076ca5580ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '684c67b8959c43f5bde0632a02d0e630'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:35.789376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2c28a6ba-db97-4d40-8445-2076ca5580ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '684c67b8959c43f5bde0632a02d0e630'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:35.801155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2c28a6ba-db97-4d40-8445-2076ca5580ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '684c67b8959c43f5bde0632a02d0e630'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:35.807598 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 965/1930] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:35 2022] DELETE /identity/v3/users/684c67b8959c43f5bde0632a02d0e630 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:35.812272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91e45a-784a-4a86-aed5-a1fbf1cc8c1a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:35.812845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91e45a-784a-4a86-aed5-a1fbf1cc8c1a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:35.812845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91e45a-784a-4a86-aed5-a1fbf1cc8c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:35.831714 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2a91e45a-784a-4a86-aed5-a1fbf1cc8c1a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:35.832881 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2a91e45a-784a-4a86-aed5-a1fbf1cc8c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:35.894249 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 966/1931] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:35.904333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28a6ba-db97-4d40-8445-2076ca5580ff admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:35.905757 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2c28a6ba-db97-4d40-8445-2076ca5580ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:35.968021 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28a6ba-db97-4d40-8445-2076ca5580ff admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:35.971781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9452f3-f284-4359-9472-3d8436b18859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:35.972417 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9452f3-f284-4359-9472-3d8436b18859 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:35.972699 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9452f3-f284-4359-9472-3d8436b18859 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:35.972983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9452f3-f284-4359-9472-3d8436b18859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:35.983945 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9452f3-f284-4359-9472-3d8436b18859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:35.985903 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9452f3-f284-4359-9472-3d8436b18859 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:35.996199 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 966/1932] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:36.288486 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91e45a-784a-4a86-aed5-a1fbf1cc8c1a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:36.290010 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2a91e45a-784a-4a86-aed5-a1fbf1cc8c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:36.302419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a91e45a-784a-4a86-aed5-a1fbf1cc8c1a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:36.307131 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce9477-070e-41db-92d3-08c5310b5239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:36.307804 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce9477-070e-41db-92d3-08c5310b5239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:36.308108 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce9477-070e-41db-92d3-08c5310b5239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:36.308399 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce9477-070e-41db-92d3-08c5310b5239 admin admin] PATH_INFO: `/v3/projects/fe184d1bd78b46d998114c0ea46793d4` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:36.309610 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce9477-070e-41db-92d3-08c5310b5239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe184d1bd78b46d998114c0ea46793d4)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:36.311708 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce9477-070e-41db-92d3-08c5310b5239 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:36.363424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c4ce9477-070e-41db-92d3-08c5310b5239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe184d1bd78b46d998114c0ea46793d4', 'request_id': 'req-c4ce9477-070e-41db-92d3-08c5310b5239'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:36.369353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c4ce9477-070e-41db-92d3-08c5310b5239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe184d1bd78b46d998114c0ea46793d4', 'request_id': 'req-c4ce9477-070e-41db-92d3-08c5310b5239'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:36.371303 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:36.378553 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c4ce9477-070e-41db-92d3-08c5310b5239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe184d1bd78b46d998114c0ea46793d4', 'request_id': 'req-c4ce9477-070e-41db-92d3-08c5310b5239'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:36.393196 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c4ce9477-070e-41db-92d3-08c5310b5239 admin admin] The token cache is being invalidate because project fe184d1bd78b46d998114c0ea46793d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:36.393490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c4ce9477-070e-41db-92d3-08c5310b5239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:36.394156 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c4ce9477-070e-41db-92d3-08c5310b5239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:36.395793 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 967/1933] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:36 2022] DELETE /identity/v3/projects/fe184d1bd78b46d998114c0ea46793d4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:36.400341 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9452f3-f284-4359-9472-3d8436b18859 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:36.410548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9452f3-f284-4359-9472-3d8436b18859 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:36.454053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:36.470874 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2b4bd-90e5-4867-b9d0-ddbd260f2ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:36.471557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2b4bd-90e5-4867-b9d0-ddbd260f2ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:36.471852 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2b4bd-90e5-4867-b9d0-ddbd260f2ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:36.472119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2b4bd-90e5-4867-b9d0-ddbd260f2ef8 admin admin] PATH_INFO: `/v3/users/a613fb99491f4e7b9c012c7422dff842` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:36.473251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2b4bd-90e5-4867-b9d0-ddbd260f2ef8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a613fb99491f4e7b9c012c7422dff842)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:36.475823 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2b4bd-90e5-4867-b9d0-ddbd260f2ef8 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:36.526938 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e3e2b4bd-90e5-4867-b9d0-ddbd260f2ef8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a613fb99491f4e7b9c012c7422dff842'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:36.534397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e3e2b4bd-90e5-4867-b9d0-ddbd260f2ef8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a613fb99491f4e7b9c012c7422dff842'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:36.535417 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e3e2b4bd-90e5-4867-b9d0-ddbd260f2ef8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a613fb99491f4e7b9c012c7422dff842'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:36.544570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e3e2b4bd-90e5-4867-b9d0-ddbd260f2ef8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a613fb99491f4e7b9c012c7422dff842'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:36.559016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e3e2b4bd-90e5-4867-b9d0-ddbd260f2ef8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a613fb99491f4e7b9c012c7422dff842'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:36.565739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 967/1934] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:36 2022] DELETE /identity/v3/users/a613fb99491f4e7b9c012c7422dff842 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:36.858169 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce9477-070e-41db-92d3-08c5310b5239 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:36.859765 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c4ce9477-070e-41db-92d3-08c5310b5239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:36.873884 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce9477-070e-41db-92d3-08c5310b5239 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:36.933660 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a1e90-44d3-4c34-9d32-d1316169d7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:36.934298 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a1e90-44d3-4c34-9d32-d1316169d7a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:36.934575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a1e90-44d3-4c34-9d32-d1316169d7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:36.934855 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a1e90-44d3-4c34-9d32-d1316169d7a4 admin admin] PATH_INFO: `/v3/projects/43c90118339543b8b7420fd319a8c9cc` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:36.936052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a1e90-44d3-4c34-9d32-d1316169d7a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43c90118339543b8b7420fd319a8c9cc)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:36.938737 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52a1e90-44d3-4c34-9d32-d1316169d7a4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:36.986730 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a52a1e90-44d3-4c34-9d32-d1316169d7a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43c90118339543b8b7420fd319a8c9cc', 'request_id': 'req-a52a1e90-44d3-4c34-9d32-d1316169d7a4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:36.992548 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a52a1e90-44d3-4c34-9d32-d1316169d7a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43c90118339543b8b7420fd319a8c9cc', 'request_id': 'req-a52a1e90-44d3-4c34-9d32-d1316169d7a4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:36.999020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a52a1e90-44d3-4c34-9d32-d1316169d7a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43c90118339543b8b7420fd319a8c9cc', 'request_id': 'req-a52a1e90-44d3-4c34-9d32-d1316169d7a4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:37.011457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a52a1e90-44d3-4c34-9d32-d1316169d7a4 admin admin] The token cache is being invalidate because project 43c90118339543b8b7420fd319a8c9cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:37.011798 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a52a1e90-44d3-4c34-9d32-d1316169d7a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:37.012534 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-a52a1e90-44d3-4c34-9d32-d1316169d7a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:37.014415 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 968/1935] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:36 2022] DELETE /identity/v3/projects/43c90118339543b8b7420fd319a8c9cc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:37.019610 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad91b6-ab50-449d-a007-6855f5d1bf72 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:37.020107 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad91b6-ab50-449d-a007-6855f5d1bf72 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:37.020214 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad91b6-ab50-449d-a007-6855f5d1bf72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:37.041423 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2cad91b6-ab50-449d-a007-6855f5d1bf72 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:37.042737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2cad91b6-ab50-449d-a007-6855f5d1bf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:37.103571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 968/1936] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:37.112957 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4bc16-552e-44ad-90ab-85761a14a2f6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:37.113269 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4bc16-552e-44ad-90ab-85761a14a2f6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:37.113544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4bc16-552e-44ad-90ab-85761a14a2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:37.135160 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f0d4bc16-552e-44ad-90ab-85761a14a2f6 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:37.136348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f0d4bc16-552e-44ad-90ab-85761a14a2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:37.145690 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 969/1937] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:37.151463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad91b6-ab50-449d-a007-6855f5d1bf72 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:37.163388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad91b6-ab50-449d-a007-6855f5d1bf72 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:37.168418 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3ea76-2bd2-4088-a0b6-e412af15e928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:37.169043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3ea76-2bd2-4088-a0b6-e412af15e928 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:37.169325 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3ea76-2bd2-4088-a0b6-e412af15e928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:37.169601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3ea76-2bd2-4088-a0b6-e412af15e928 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:37.170262 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3ea76-2bd2-4088-a0b6-e412af15e928 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:37.170564 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3ea76-2bd2-4088-a0b6-e412af15e928 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:37.172200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3ea76-2bd2-4088-a0b6-e412af15e928 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:37.185808 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 969/1938] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:13:37.189926 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4bc16-552e-44ad-90ab-85761a14a2f6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:37.201899 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4bc16-552e-44ad-90ab-85761a14a2f6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:37.206515 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d17dbb-e0b2-48ff-ac9e-10bf938ed57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:37.207155 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d17dbb-e0b2-48ff-ac9e-10bf938ed57a admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:37.207430 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d17dbb-e0b2-48ff-ac9e-10bf938ed57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:37.207697 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d17dbb-e0b2-48ff-ac9e-10bf938ed57a admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:37.208337 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d17dbb-e0b2-48ff-ac9e-10bf938ed57a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:37.211576 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d17dbb-e0b2-48ff-ac9e-10bf938ed57a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:37.244708 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 970/1939] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:37 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:37.249016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3ea76-2bd2-4088-a0b6-e412af15e928 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:37.266784 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3ea76-2bd2-4088-a0b6-e412af15e928 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:37.334893 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:37.348442 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f9a73-d03e-4a67-b8a9-ef842ba9f04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:37.349057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f9a73-d03e-4a67-b8a9-ef842ba9f04e admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:37.349336 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f9a73-d03e-4a67-b8a9-ef842ba9f04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:37.349635 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f9a73-d03e-4a67-b8a9-ef842ba9f04e admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:37.350286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f9a73-d03e-4a67-b8a9-ef842ba9f04e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:37.352220 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f9a73-d03e-4a67-b8a9-ef842ba9f04e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:37.376672 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 970/1940] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:37 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:37.378206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d17dbb-e0b2-48ff-ac9e-10bf938ed57a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:37.389573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d17dbb-e0b2-48ff-ac9e-10bf938ed57a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:37.394258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099ed54-b235-4200-ac9d-be0b71cbe04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:37.394889 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099ed54-b235-4200-ac9d-be0b71cbe04d admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:37.395179 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099ed54-b235-4200-ac9d-be0b71cbe04d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:37.395458 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099ed54-b235-4200-ac9d-be0b71cbe04d admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:37.396103 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099ed54-b235-4200-ac9d-be0b71cbe04d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:37.398397 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099ed54-b235-4200-ac9d-be0b71cbe04d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:37.409752 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 971/1941] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:37.413716 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f9a73-d03e-4a67-b8a9-ef842ba9f04e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:37.423888 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f9a73-d03e-4a67-b8a9-ef842ba9f04e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:37.428740 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e84e568-9fbb-4979-bf81-ce8c8f96c9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:37.429425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84e568-9fbb-4979-bf81-ce8c8f96c9c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:37.429638 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84e568-9fbb-4979-bf81-ce8c8f96c9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:37.429917 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e84e568-9fbb-4979-bf81-ce8c8f96c9c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:37.430560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84e568-9fbb-4979-bf81-ce8c8f96c9c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:37.432463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e84e568-9fbb-4979-bf81-ce8c8f96c9c0 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:37.445250 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 971/1942] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:37.449299 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099ed54-b235-4200-ac9d-be0b71cbe04d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:37.462355 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099ed54-b235-4200-ac9d-be0b71cbe04d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:37.467011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c0972-ddd1-491d-8704-92ef79500b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:37.467659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c0972-ddd1-491d-8704-92ef79500b80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:37.467967 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c0972-ddd1-491d-8704-92ef79500b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:37.468241 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c0972-ddd1-491d-8704-92ef79500b80 admin admin] PATH_INFO: `/v3/projects/86559de0c86b41cdac3947acd0b05d5f/users/617af6ec5da04ad7ac34e7af7fe0450f/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:37.484490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c0972-ddd1-491d-8704-92ef79500b80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86559de0c86b41cdac3947acd0b05d5f, user_id=617af6ec5da04ad7ac34e7af7fe0450f, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:37.487475 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c0972-ddd1-491d-8704-92ef79500b80 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:37.500922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 972/1943] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:37 2022] PUT /identity/v3/projects/86559de0c86b41cdac3947acd0b05d5f/users/617af6ec5da04ad7ac34e7af7fe0450f/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:37.512455 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d8f9f-4a01-4f3e-be17-cc083e43c0f0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:37.512762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d8f9f-4a01-4f3e-be17-cc083e43c0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:37.513036 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d8f9f-4a01-4f3e-be17-cc083e43c0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:37.546270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9c9d8f9f-4a01-4f3e-be17-cc083e43c0f0 None None] MFA Rules not processed for user `617af6ec5da04ad7ac34e7af7fe0450f`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:37.547511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9c9d8f9f-4a01-4f3e-be17-cc083e43c0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:37.615788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 972/1944] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:37.620537 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c0972-ddd1-491d-8704-92ef79500b80 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:37.638815 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:37.688678 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c0972-ddd1-491d-8704-92ef79500b80 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:37.696416 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad2f51-63ba-4041-920b-7cc2c17295b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:37.697065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad2f51-63ba-4041-920b-7cc2c17295b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:37.697347 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad2f51-63ba-4041-920b-7cc2c17295b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:37.697619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad2f51-63ba-4041-920b-7cc2c17295b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:37.698274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad2f51-63ba-4041-920b-7cc2c17295b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:37.700254 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad2f51-63ba-4041-920b-7cc2c17295b1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:37.732150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 973/1945] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:37 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:37.733290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d8f9f-4a01-4f3e-be17-cc083e43c0f0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:37.744765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d8f9f-4a01-4f3e-be17-cc083e43c0f0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:37.809674 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bd9a0-209f-4cf7-b76d-c7c98353e948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:37.810363 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bd9a0-209f-4cf7-b76d-c7c98353e948 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:37.810746 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bd9a0-209f-4cf7-b76d-c7c98353e948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:37.811082 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bd9a0-209f-4cf7-b76d-c7c98353e948 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:37.811593 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bd9a0-209f-4cf7-b76d-c7c98353e948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:37.813552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bd9a0-209f-4cf7-b76d-c7c98353e948 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:37.839499 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 973/1946] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:37 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:13:37.843487 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad2f51-63ba-4041-920b-7cc2c17295b1 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:37.854736 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad2f51-63ba-4041-920b-7cc2c17295b1 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:37.859398 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d521205-8a1e-4951-9bbd-1d84d9eeed68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:37.859984 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d521205-8a1e-4951-9bbd-1d84d9eeed68 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:37.860261 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d521205-8a1e-4951-9bbd-1d84d9eeed68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:37.860532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d521205-8a1e-4951-9bbd-1d84d9eeed68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:37.861173 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d521205-8a1e-4951-9bbd-1d84d9eeed68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:37.862769 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d521205-8a1e-4951-9bbd-1d84d9eeed68 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:37.874394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 974/1947] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:37.878697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bd9a0-209f-4cf7-b76d-c7c98353e948 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:37.892755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bd9a0-209f-4cf7-b76d-c7c98353e948 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:37.898472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ca006-8637-4df2-bf6b-ef05117ae96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:37.899242 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ca006-8637-4df2-bf6b-ef05117ae96f admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:37.899553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ca006-8637-4df2-bf6b-ef05117ae96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:37.899853 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ca006-8637-4df2-bf6b-ef05117ae96f admin admin] PATH_INFO: `/v3/projects/3e5b1a0382a14a2eba294c6acfec72b6/users/0075df9cb0e842658c61863f8fc5a583/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:37.916953 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ca006-8637-4df2-bf6b-ef05117ae96f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e5b1a0382a14a2eba294c6acfec72b6, user_id=0075df9cb0e842658c61863f8fc5a583, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:37.919897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ca006-8637-4df2-bf6b-ef05117ae96f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:37.932789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 974/1948] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:37 2022] PUT /identity/v3/projects/3e5b1a0382a14a2eba294c6acfec72b6/users/0075df9cb0e842658c61863f8fc5a583/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:37.937074 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d521205-8a1e-4951-9bbd-1d84d9eeed68 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:37.948062 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d521205-8a1e-4951-9bbd-1d84d9eeed68 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:38.010582 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac6b80-ca24-4b52-8683-7b2b47d13fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:38.011214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac6b80-ca24-4b52-8683-7b2b47d13fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:38.011491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac6b80-ca24-4b52-8683-7b2b47d13fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:38.011759 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac6b80-ca24-4b52-8683-7b2b47d13fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:38.012386 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac6b80-ca24-4b52-8683-7b2b47d13fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:38.014646 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac6b80-ca24-4b52-8683-7b2b47d13fb5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:38.027769 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 975/1949] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:38.032316 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ca006-8637-4df2-bf6b-ef05117ae96f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:38.043172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ca006-8637-4df2-bf6b-ef05117ae96f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:38.049754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99a4be-1cd2-4a2a-b8f1-b3c4ee55d2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:38.050742 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99a4be-1cd2-4a2a-b8f1-b3c4ee55d2a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:38.051152 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99a4be-1cd2-4a2a-b8f1-b3c4ee55d2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:38.051431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99a4be-1cd2-4a2a-b8f1-b3c4ee55d2a9 admin admin] PATH_INFO: `/v3/projects/3e5b1a0382a14a2eba294c6acfec72b6/users/0075df9cb0e842658c61863f8fc5a583/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:38.053408 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99a4be-1cd2-4a2a-b8f1-b3c4ee55d2a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e5b1a0382a14a2eba294c6acfec72b6, user_id=0075df9cb0e842658c61863f8fc5a583, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:38.056144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99a4be-1cd2-4a2a-b8f1-b3c4ee55d2a9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:38.066778 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 975/1950] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:38 2022] PUT /identity/v3/projects/3e5b1a0382a14a2eba294c6acfec72b6/users/0075df9cb0e842658c61863f8fc5a583/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:38.079131 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cce88-507c-481a-a2b2-e9bf7fe0afa2 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:38.079415 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cce88-507c-481a-a2b2-e9bf7fe0afa2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:38.079686 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cce88-507c-481a-a2b2-e9bf7fe0afa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:38.109459 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a71cce88-507c-481a-a2b2-e9bf7fe0afa2 None None] MFA Rules not processed for user `0075df9cb0e842658c61863f8fc5a583`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:38.110668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a71cce88-507c-481a-a2b2-e9bf7fe0afa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:38.185913 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 976/1951] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:38 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:38.197517 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99a4be-1cd2-4a2a-b8f1-b3c4ee55d2a9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:38.199511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-bb99a4be-1cd2-4a2a-b8f1-b3c4ee55d2a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:38.218883 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:38.268947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99a4be-1cd2-4a2a-b8f1-b3c4ee55d2a9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:38.273445 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5501c-5d91-479b-bedf-8bc24a9ea957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:38.274089 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5501c-5d91-479b-bedf-8bc24a9ea957 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:38.274356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5501c-5d91-479b-bedf-8bc24a9ea957 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:38.274635 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5501c-5d91-479b-bedf-8bc24a9ea957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:38.287492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5501c-5d91-479b-bedf-8bc24a9ea957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:38.288990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5501c-5d91-479b-bedf-8bc24a9ea957 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:38.299091 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 976/1952] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:38 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:38.397290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cce88-507c-481a-a2b2-e9bf7fe0afa2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:38.407512 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cce88-507c-481a-a2b2-e9bf7fe0afa2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:38.412836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26272ed-bad1-4dea-9a5a-caede5f47f36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:38.413556 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26272ed-bad1-4dea-9a5a-caede5f47f36 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:38.413996 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26272ed-bad1-4dea-9a5a-caede5f47f36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:38.414336 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26272ed-bad1-4dea-9a5a-caede5f47f36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:38.423618 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:38.484835 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26272ed-bad1-4dea-9a5a-caede5f47f36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:38.486286 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26272ed-bad1-4dea-9a5a-caede5f47f36 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:38.496485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 977/1953] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:38.789506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5501c-5d91-479b-bedf-8bc24a9ea957 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:38.869620 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5501c-5d91-479b-bedf-8bc24a9ea957 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:38.875088 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391d6d9-07f7-44a4-b2e2-8ff3d1971483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:38.875738 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391d6d9-07f7-44a4-b2e2-8ff3d1971483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:38.875738 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391d6d9-07f7-44a4-b2e2-8ff3d1971483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:38.875738 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391d6d9-07f7-44a4-b2e2-8ff3d1971483 admin admin] PATH_INFO: `/v3/users/617af6ec5da04ad7ac34e7af7fe0450f` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:38.877015 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391d6d9-07f7-44a4-b2e2-8ff3d1971483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=617af6ec5da04ad7ac34e7af7fe0450f)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:38.879014 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391d6d9-07f7-44a4-b2e2-8ff3d1971483 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:38.943560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e391d6d9-07f7-44a4-b2e2-8ff3d1971483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '617af6ec5da04ad7ac34e7af7fe0450f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:38.954092 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e391d6d9-07f7-44a4-b2e2-8ff3d1971483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '617af6ec5da04ad7ac34e7af7fe0450f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:38.954092 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e391d6d9-07f7-44a4-b2e2-8ff3d1971483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '617af6ec5da04ad7ac34e7af7fe0450f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:38.964223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e391d6d9-07f7-44a4-b2e2-8ff3d1971483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '617af6ec5da04ad7ac34e7af7fe0450f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:38.978535 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e391d6d9-07f7-44a4-b2e2-8ff3d1971483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '617af6ec5da04ad7ac34e7af7fe0450f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:38.985976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 977/1954] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:38 2022] DELETE /identity/v3/users/617af6ec5da04ad7ac34e7af7fe0450f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:38.991516 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb37939-99c8-441c-b147-a4cafbc9c5cc None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:38.991985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb37939-99c8-441c-b147-a4cafbc9c5cc None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:38.992193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb37939-99c8-441c-b147-a4cafbc9c5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:39.015739 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-eeb37939-99c8-441c-b147-a4cafbc9c5cc None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:39.016982 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-eeb37939-99c8-441c-b147-a4cafbc9c5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:39.086956 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 978/1955] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:39.096637 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391d6d9-07f7-44a4-b2e2-8ff3d1971483 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:39.098067 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e391d6d9-07f7-44a4-b2e2-8ff3d1971483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:39.134817 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:39.159994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391d6d9-07f7-44a4-b2e2-8ff3d1971483 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:39.164625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4d4b6-94dc-498b-b980-fbcb9e39cd98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:39.165243 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4d4b6-94dc-498b-b980-fbcb9e39cd98 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:39.165524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4d4b6-94dc-498b-b980-fbcb9e39cd98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:39.165800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4d4b6-94dc-498b-b980-fbcb9e39cd98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:39.174605 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4d4b6-94dc-498b-b980-fbcb9e39cd98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:39.176103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4d4b6-94dc-498b-b980-fbcb9e39cd98 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:39.185542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 978/1956] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:39.483414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb37939-99c8-441c-b147-a4cafbc9c5cc None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:39.484916 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-eeb37939-99c8-441c-b147-a4cafbc9c5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:39.497882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb37939-99c8-441c-b147-a4cafbc9c5cc None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:39.502684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01125e04-8b33-4534-baef-70927f5652da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:39.503355 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01125e04-8b33-4534-baef-70927f5652da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:39.503639 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01125e04-8b33-4534-baef-70927f5652da admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:39.503918 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01125e04-8b33-4534-baef-70927f5652da admin admin] PATH_INFO: `/v3/projects/86559de0c86b41cdac3947acd0b05d5f` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:39.505053 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01125e04-8b33-4534-baef-70927f5652da admin admin] RBAC: Authorizing `identity:delete_project(project_id=86559de0c86b41cdac3947acd0b05d5f)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:39.507039 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01125e04-8b33-4534-baef-70927f5652da admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:39.563467 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-01125e04-8b33-4534-baef-70927f5652da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86559de0c86b41cdac3947acd0b05d5f', 'request_id': 'req-01125e04-8b33-4534-baef-70927f5652da'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:39.569883 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-01125e04-8b33-4534-baef-70927f5652da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86559de0c86b41cdac3947acd0b05d5f', 'request_id': 'req-01125e04-8b33-4534-baef-70927f5652da'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:39.576886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-01125e04-8b33-4534-baef-70927f5652da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86559de0c86b41cdac3947acd0b05d5f', 'request_id': 'req-01125e04-8b33-4534-baef-70927f5652da'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:39.592192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-01125e04-8b33-4534-baef-70927f5652da admin admin] The token cache is being invalidate because project 86559de0c86b41cdac3947acd0b05d5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:39.592635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-01125e04-8b33-4534-baef-70927f5652da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:39.593409 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-01125e04-8b33-4534-baef-70927f5652da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:39.595134 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 979/1957] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:39 2022] DELETE /identity/v3/projects/86559de0c86b41cdac3947acd0b05d5f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:39.599124 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4d4b6-94dc-498b-b980-fbcb9e39cd98 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:39.610011 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4d4b6-94dc-498b-b980-fbcb9e39cd98 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:39.689536 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37001a56-2941-4e63-b13a-8b0dcebb72e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:39.690183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37001a56-2941-4e63-b13a-8b0dcebb72e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:39.690463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37001a56-2941-4e63-b13a-8b0dcebb72e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:39.690734 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37001a56-2941-4e63-b13a-8b0dcebb72e1 admin admin] PATH_INFO: `/v3/users/0075df9cb0e842658c61863f8fc5a583` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:39.691899 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37001a56-2941-4e63-b13a-8b0dcebb72e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0075df9cb0e842658c61863f8fc5a583)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:39.694100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37001a56-2941-4e63-b13a-8b0dcebb72e1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:39.747125 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-37001a56-2941-4e63-b13a-8b0dcebb72e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0075df9cb0e842658c61863f8fc5a583'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:39.754075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-37001a56-2941-4e63-b13a-8b0dcebb72e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0075df9cb0e842658c61863f8fc5a583'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:39.754758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-37001a56-2941-4e63-b13a-8b0dcebb72e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0075df9cb0e842658c61863f8fc5a583'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:39.762056 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-37001a56-2941-4e63-b13a-8b0dcebb72e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0075df9cb0e842658c61863f8fc5a583'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:39.775801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-37001a56-2941-4e63-b13a-8b0dcebb72e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0075df9cb0e842658c61863f8fc5a583'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:39.782317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 979/1958] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:39 2022] DELETE /identity/v3/users/0075df9cb0e842658c61863f8fc5a583 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:40.063774 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01125e04-8b33-4534-baef-70927f5652da admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:40.069996 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-01125e04-8b33-4534-baef-70927f5652da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:40.086796 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01125e04-8b33-4534-baef-70927f5652da admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:40.137939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:40.178473 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623efd65-1e43-4c91-b6ee-594c5134bce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:40.179635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623efd65-1e43-4c91-b6ee-594c5134bce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:40.180126 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623efd65-1e43-4c91-b6ee-594c5134bce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:40.180579 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623efd65-1e43-4c91-b6ee-594c5134bce3 admin admin] PATH_INFO: `/v3/projects/3e5b1a0382a14a2eba294c6acfec72b6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:40.182377 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623efd65-1e43-4c91-b6ee-594c5134bce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e5b1a0382a14a2eba294c6acfec72b6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:40.184611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623efd65-1e43-4c91-b6ee-594c5134bce3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:40.248483 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-623efd65-1e43-4c91-b6ee-594c5134bce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e5b1a0382a14a2eba294c6acfec72b6', 'request_id': 'req-623efd65-1e43-4c91-b6ee-594c5134bce3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:40.254397 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-623efd65-1e43-4c91-b6ee-594c5134bce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e5b1a0382a14a2eba294c6acfec72b6', 'request_id': 'req-623efd65-1e43-4c91-b6ee-594c5134bce3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:40.264251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-623efd65-1e43-4c91-b6ee-594c5134bce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e5b1a0382a14a2eba294c6acfec72b6', 'request_id': 'req-623efd65-1e43-4c91-b6ee-594c5134bce3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:40.280592 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-623efd65-1e43-4c91-b6ee-594c5134bce3 admin admin] The token cache is being invalidate because project 3e5b1a0382a14a2eba294c6acfec72b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:40.281177 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-623efd65-1e43-4c91-b6ee-594c5134bce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:40.282043 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-623efd65-1e43-4c91-b6ee-594c5134bce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:40.284084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 980/1959] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:40 2022] DELETE /identity/v3/projects/3e5b1a0382a14a2eba294c6acfec72b6 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:40.289803 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b40bdf-b7b1-45a0-acf7-34f6e6d3024a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:40.290200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b40bdf-b7b1-45a0-acf7-34f6e6d3024a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:40.290406 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b40bdf-b7b1-45a0-acf7-34f6e6d3024a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:40.311345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-84b40bdf-b7b1-45a0-acf7-34f6e6d3024a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:40.312836 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-84b40bdf-b7b1-45a0-acf7-34f6e6d3024a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:40.394471 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:40.420632 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97073495-5380-49e9-9573-064b3b54857a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:40.420928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97073495-5380-49e9-9573-064b3b54857a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:40.421193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97073495-5380-49e9-9573-064b3b54857a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:40.426336 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 980/1960] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:40.441521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-97073495-5380-49e9-9573-064b3b54857a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:40.442828 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-97073495-5380-49e9-9573-064b3b54857a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:40.451787 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 981/1961] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:40.455962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b40bdf-b7b1-45a0-acf7-34f6e6d3024a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:40.466272 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b40bdf-b7b1-45a0-acf7-34f6e6d3024a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:40.471221 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4db851-0edc-403d-9dfa-e593549b0695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:40.471833 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4db851-0edc-403d-9dfa-e593549b0695 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:40.472113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4db851-0edc-403d-9dfa-e593549b0695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:40.472389 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4db851-0edc-403d-9dfa-e593549b0695 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:40.473036 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4db851-0edc-403d-9dfa-e593549b0695 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:40.473336 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4db851-0edc-403d-9dfa-e593549b0695 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:40.475001 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4db851-0edc-403d-9dfa-e593549b0695 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:40.486603 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 981/1962] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:13:40.491058 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97073495-5380-49e9-9573-064b3b54857a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:40.500107 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97073495-5380-49e9-9573-064b3b54857a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:40.504711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13be24-7ad7-4946-86d5-7aee37f146fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:40.505244 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13be24-7ad7-4946-86d5-7aee37f146fd admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:40.505519 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13be24-7ad7-4946-86d5-7aee37f146fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:40.506321 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13be24-7ad7-4946-86d5-7aee37f146fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:40.507212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13be24-7ad7-4946-86d5-7aee37f146fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:40.509105 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13be24-7ad7-4946-86d5-7aee37f146fd admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:40.535801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 982/1963] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:40 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:40.540008 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4db851-0edc-403d-9dfa-e593549b0695 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:40.550224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4db851-0edc-403d-9dfa-e593549b0695 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:40.605353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf17dd6-e359-4c2c-a41c-8b80f480736a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:40.605957 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf17dd6-e359-4c2c-a41c-8b80f480736a admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:40.606233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf17dd6-e359-4c2c-a41c-8b80f480736a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:40.606504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf17dd6-e359-4c2c-a41c-8b80f480736a admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:40.607165 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf17dd6-e359-4c2c-a41c-8b80f480736a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:40.609057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf17dd6-e359-4c2c-a41c-8b80f480736a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:40.630272 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 982/1964] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:40 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:40.635387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13be24-7ad7-4946-86d5-7aee37f146fd admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:40.645543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13be24-7ad7-4946-86d5-7aee37f146fd admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:40.650092 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d4545-3606-4891-b909-75668c2a5c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:40.650684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d4545-3606-4891-b909-75668c2a5c42 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:40.650991 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d4545-3606-4891-b909-75668c2a5c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:40.651260 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d4545-3606-4891-b909-75668c2a5c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:40.651897 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d4545-3606-4891-b909-75668c2a5c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:40.653736 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d4545-3606-4891-b909-75668c2a5c42 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:40.663622 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 983/1965] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:40.667627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf17dd6-e359-4c2c-a41c-8b80f480736a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:40.678538 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf17dd6-e359-4c2c-a41c-8b80f480736a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:40.683838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe25bd3-b44c-4d74-b76d-4ab150f4f26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:40.684568 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe25bd3-b44c-4d74-b76d-4ab150f4f26c admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:40.684854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe25bd3-b44c-4d74-b76d-4ab150f4f26c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:40.685129 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe25bd3-b44c-4d74-b76d-4ab150f4f26c admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:40.685774 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe25bd3-b44c-4d74-b76d-4ab150f4f26c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:40.687540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe25bd3-b44c-4d74-b76d-4ab150f4f26c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:40.699605 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 983/1966] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:40.703715 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d4545-3606-4891-b909-75668c2a5c42 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:40.713715 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d4545-3606-4891-b909-75668c2a5c42 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:40.718684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f17ea-7688-4a49-8056-d2d6208764c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:40.719390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f17ea-7688-4a49-8056-d2d6208764c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:40.719672 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f17ea-7688-4a49-8056-d2d6208764c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:40.719945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f17ea-7688-4a49-8056-d2d6208764c1 admin admin] PATH_INFO: `/v3/projects/7e35f509d0cb4826b1639a17018d29a6/users/29278a807587412b98477436e8312e67/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:40.735032 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f17ea-7688-4a49-8056-d2d6208764c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e35f509d0cb4826b1639a17018d29a6, user_id=29278a807587412b98477436e8312e67, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:40.737942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f17ea-7688-4a49-8056-d2d6208764c1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:40.752455 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 984/1967] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:40 2022] PUT /identity/v3/projects/7e35f509d0cb4826b1639a17018d29a6/users/29278a807587412b98477436e8312e67/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:40.764234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf833938-1951-476a-b2fb-4b49aa6a872d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:40.764474 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf833938-1951-476a-b2fb-4b49aa6a872d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:40.764834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf833938-1951-476a-b2fb-4b49aa6a872d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:40.797355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-bf833938-1951-476a-b2fb-4b49aa6a872d None None] MFA Rules not processed for user `29278a807587412b98477436e8312e67`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:40.798571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-bf833938-1951-476a-b2fb-4b49aa6a872d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:40.866474 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 984/1968] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:40.871193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f17ea-7688-4a49-8056-d2d6208764c1 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:40.939996 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f17ea-7688-4a49-8056-d2d6208764c1 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:40.945700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9943e6-518c-4c70-aad4-4250eae03eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:40.946412 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9943e6-518c-4c70-aad4-4250eae03eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:40.946768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9943e6-518c-4c70-aad4-4250eae03eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:40.947081 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9943e6-518c-4c70-aad4-4250eae03eb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:40.947775 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9943e6-518c-4c70-aad4-4250eae03eb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:40.949715 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9943e6-518c-4c70-aad4-4250eae03eb0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:40.954596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:40.982407 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 985/1969] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:40 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:13:40.986678 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf833938-1951-476a-b2fb-4b49aa6a872d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:40.999318 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf833938-1951-476a-b2fb-4b49aa6a872d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:41.048657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:41.060105 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c24053-92c9-4989-8281-17e442b2f5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:41.060730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c24053-92c9-4989-8281-17e442b2f5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:41.061044 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c24053-92c9-4989-8281-17e442b2f5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:41.061282 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c24053-92c9-4989-8281-17e442b2f5b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:41.061920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c24053-92c9-4989-8281-17e442b2f5b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:41.063915 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c24053-92c9-4989-8281-17e442b2f5b6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:41.086086 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 985/1970] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:40 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:41.090753 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9943e6-518c-4c70-aad4-4250eae03eb0 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:41.101508 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9943e6-518c-4c70-aad4-4250eae03eb0 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:41.106046 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7c7ff-c3a3-4caf-9811-7154571455c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:41.106644 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7c7ff-c3a3-4caf-9811-7154571455c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:41.106954 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7c7ff-c3a3-4caf-9811-7154571455c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:41.107228 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7c7ff-c3a3-4caf-9811-7154571455c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:41.107865 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb7c7ff-c3a3-4caf-9811-7154571455c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:41.109457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb7c7ff-c3a3-4caf-9811-7154571455c3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:41.120021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 986/1971] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:41.124055 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c24053-92c9-4989-8281-17e442b2f5b6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:41.134136 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c24053-92c9-4989-8281-17e442b2f5b6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:41.138852 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de3acf-9d74-4733-a182-e3b03498e33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:41.139484 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de3acf-9d74-4733-a182-e3b03498e33a admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:41.139765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de3acf-9d74-4733-a182-e3b03498e33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:41.140038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de3acf-9d74-4733-a182-e3b03498e33a admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:41.140688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de3acf-9d74-4733-a182-e3b03498e33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:41.142308 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de3acf-9d74-4733-a182-e3b03498e33a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:41.152597 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 986/1972] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:41.156928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7c7ff-c3a3-4caf-9811-7154571455c3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:41.167579 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7c7ff-c3a3-4caf-9811-7154571455c3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:41.172557 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a059c-ea96-4217-8b28-e3cc7aadfbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:41.173226 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a059c-ea96-4217-8b28-e3cc7aadfbbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:41.173535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a059c-ea96-4217-8b28-e3cc7aadfbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:41.173813 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a059c-ea96-4217-8b28-e3cc7aadfbbe admin admin] PATH_INFO: `/v3/projects/ce68220ead6246258e326e20b93c37ba/users/fe743b8696814f1da8fb13b4e9f31c82/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:41.189420 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a059c-ea96-4217-8b28-e3cc7aadfbbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce68220ead6246258e326e20b93c37ba, user_id=fe743b8696814f1da8fb13b4e9f31c82, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:41.192228 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a059c-ea96-4217-8b28-e3cc7aadfbbe admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:41.205274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 987/1973] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:41 2022] PUT /identity/v3/projects/ce68220ead6246258e326e20b93c37ba/users/fe743b8696814f1da8fb13b4e9f31c82/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:41.215965 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a88c84-6cea-43b1-a7a3-7a33950aa095 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:41.216255 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a88c84-6cea-43b1-a7a3-7a33950aa095 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:41.216530 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a88c84-6cea-43b1-a7a3-7a33950aa095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:41.245009 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-48a88c84-6cea-43b1-a7a3-7a33950aa095 None None] MFA Rules not processed for user `fe743b8696814f1da8fb13b4e9f31c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:41.246223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-48a88c84-6cea-43b1-a7a3-7a33950aa095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:41.312237 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 987/1974] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:41.316735 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a059c-ea96-4217-8b28-e3cc7aadfbbe admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:41.383630 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a059c-ea96-4217-8b28-e3cc7aadfbbe admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:41.388314 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e1911-8013-4f9a-b8b0-82a4d0eba4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:41.389516 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e1911-8013-4f9a-b8b0-82a4d0eba4ce admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:41.389802 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e1911-8013-4f9a-b8b0-82a4d0eba4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:41.390075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e1911-8013-4f9a-b8b0-82a4d0eba4ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:41.390724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e1911-8013-4f9a-b8b0-82a4d0eba4ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:41.392686 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e1911-8013-4f9a-b8b0-82a4d0eba4ce admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:41.422914 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 988/1975] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:41 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:41.426539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a88c84-6cea-43b1-a7a3-7a33950aa095 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:41.439654 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a88c84-6cea-43b1-a7a3-7a33950aa095 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:41.513983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb6ad8-d900-4e2f-bffc-93fcf0f7e145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:41.514603 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb6ad8-d900-4e2f-bffc-93fcf0f7e145 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:41.514913 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb6ad8-d900-4e2f-bffc-93fcf0f7e145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:41.515194 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb6ad8-d900-4e2f-bffc-93fcf0f7e145 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:41.515843 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb6ad8-d900-4e2f-bffc-93fcf0f7e145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:41.517758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb6ad8-d900-4e2f-bffc-93fcf0f7e145 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:41.538813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 988/1976] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:41 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:41.543017 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e1911-8013-4f9a-b8b0-82a4d0eba4ce admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:41.554089 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e1911-8013-4f9a-b8b0-82a4d0eba4ce admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:41.558536 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcc482-2008-4381-972a-21a83a752d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:41.559156 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcc482-2008-4381-972a-21a83a752d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:41.559434 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcc482-2008-4381-972a-21a83a752d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:41.559707 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcc482-2008-4381-972a-21a83a752d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:41.560341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcc482-2008-4381-972a-21a83a752d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:41.561935 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcc482-2008-4381-972a-21a83a752d39 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:41.573039 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 989/1977] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:41.577130 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb6ad8-d900-4e2f-bffc-93fcf0f7e145 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:41.587606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb6ad8-d900-4e2f-bffc-93fcf0f7e145 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:41.592307 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7519423-30c3-406f-8cf5-fd9098ee06d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:41.592953 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7519423-30c3-406f-8cf5-fd9098ee06d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:41.593235 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7519423-30c3-406f-8cf5-fd9098ee06d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:41.593512 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7519423-30c3-406f-8cf5-fd9098ee06d9 admin admin] PATH_INFO: `/v3/projects/d557f4d5eb8a4d958fc196a1008e9da0/users/0d3faa0d52ed4989ab3dbe445231ca48/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:41.608324 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7519423-30c3-406f-8cf5-fd9098ee06d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d557f4d5eb8a4d958fc196a1008e9da0, user_id=0d3faa0d52ed4989ab3dbe445231ca48, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:41.610993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7519423-30c3-406f-8cf5-fd9098ee06d9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:41.623787 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 989/1978] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:41 2022] PUT /identity/v3/projects/d557f4d5eb8a4d958fc196a1008e9da0/users/0d3faa0d52ed4989ab3dbe445231ca48/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:41.627856 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcc482-2008-4381-972a-21a83a752d39 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:41.638057 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcc482-2008-4381-972a-21a83a752d39 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:41.694572 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72226d23-65e4-4c9f-b026-c5ce7a7b34ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:41.695193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72226d23-65e4-4c9f-b026-c5ce7a7b34ed admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:41.695468 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72226d23-65e4-4c9f-b026-c5ce7a7b34ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:41.695739 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72226d23-65e4-4c9f-b026-c5ce7a7b34ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:41.696372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72226d23-65e4-4c9f-b026-c5ce7a7b34ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:41.697943 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72226d23-65e4-4c9f-b026-c5ce7a7b34ed admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:41.707287 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 990/1979] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:41.712396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7519423-30c3-406f-8cf5-fd9098ee06d9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:41.722392 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7519423-30c3-406f-8cf5-fd9098ee06d9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:41.727316 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a8a30-00dc-4af8-aaa6-48271b2f5ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:41.727965 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a8a30-00dc-4af8-aaa6-48271b2f5ef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:41.728244 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a8a30-00dc-4af8-aaa6-48271b2f5ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:41.728516 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a8a30-00dc-4af8-aaa6-48271b2f5ef3 admin admin] PATH_INFO: `/v3/projects/d557f4d5eb8a4d958fc196a1008e9da0/users/0d3faa0d52ed4989ab3dbe445231ca48/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:41.730939 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a8a30-00dc-4af8-aaa6-48271b2f5ef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d557f4d5eb8a4d958fc196a1008e9da0, user_id=0d3faa0d52ed4989ab3dbe445231ca48, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:41.736317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a8a30-00dc-4af8-aaa6-48271b2f5ef3 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:41.747542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 990/1980] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:41 2022] PUT /identity/v3/projects/d557f4d5eb8a4d958fc196a1008e9da0/users/0d3faa0d52ed4989ab3dbe445231ca48/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:41.759456 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52338b4-8a67-49b8-b89b-7afe659aeca4 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:41.759741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52338b4-8a67-49b8-b89b-7afe659aeca4 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:41.760035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52338b4-8a67-49b8-b89b-7afe659aeca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:41.789376 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a52338b4-8a67-49b8-b89b-7afe659aeca4 None None] MFA Rules not processed for user `0d3faa0d52ed4989ab3dbe445231ca48`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:41.790620 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a52338b4-8a67-49b8-b89b-7afe659aeca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:41.798954 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:41.860115 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 991/1981] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:41.871603 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a8a30-00dc-4af8-aaa6-48271b2f5ef3 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:41.873339 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-739a8a30-00dc-4af8-aaa6-48271b2f5ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:41.887983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:41.933474 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a8a30-00dc-4af8-aaa6-48271b2f5ef3 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:41.937140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495c291-0882-47ee-b488-1a3a1617e411 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:41.937745 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495c291-0882-47ee-b488-1a3a1617e411 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:41.938025 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495c291-0882-47ee-b488-1a3a1617e411 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:41.938298 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495c291-0882-47ee-b488-1a3a1617e411 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:42.000130 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495c291-0882-47ee-b488-1a3a1617e411 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:42.001618 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495c291-0882-47ee-b488-1a3a1617e411 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:42.011787 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 991/1982] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:43.788857 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52338b4-8a67-49b8-b89b-7afe659aeca4 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:43.790338 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a52338b4-8a67-49b8-b89b-7afe659aeca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:43.803694 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52338b4-8a67-49b8-b89b-7afe659aeca4 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:43.809454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555af6a6-abd5-490d-b1dd-bdb4c035ab5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:43.810137 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555af6a6-abd5-490d-b1dd-bdb4c035ab5b service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:43.810423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555af6a6-abd5-490d-b1dd-bdb4c035ab5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:43.810697 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555af6a6-abd5-490d-b1dd-bdb4c035ab5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:43.881023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555af6a6-abd5-490d-b1dd-bdb4c035ab5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:43.882479 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555af6a6-abd5-490d-b1dd-bdb4c035ab5b service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:43.893180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 992/1983] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:13:45.908730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495c291-0882-47ee-b488-1a3a1617e411 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:45.917733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495c291-0882-47ee-b488-1a3a1617e411 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:45.921387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7390f-24e8-4686-9ced-ec89a41e1dfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:45.921984 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7390f-24e8-4686-9ced-ec89a41e1dfe service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:45.922260 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7390f-24e8-4686-9ced-ec89a41e1dfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:45.922531 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7390f-24e8-4686-9ced-ec89a41e1dfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:45.932946 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7390f-24e8-4686-9ced-ec89a41e1dfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:45.934470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7390f-24e8-4686-9ced-ec89a41e1dfe service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:45.944282 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 992/1984] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:51.348017 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555af6a6-abd5-490d-b1dd-bdb4c035ab5b service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:51.426249 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555af6a6-abd5-490d-b1dd-bdb4c035ab5b service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:51.431710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8df5e-245d-4ae5-85df-533468bce888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:51.432384 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8df5e-245d-4ae5-85df-533468bce888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:51.432671 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8df5e-245d-4ae5-85df-533468bce888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:51.433096 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8df5e-245d-4ae5-85df-533468bce888 admin admin] PATH_INFO: `/v3/users/29278a807587412b98477436e8312e67` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:51.434426 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8df5e-245d-4ae5-85df-533468bce888 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29278a807587412b98477436e8312e67)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:51.436411 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8df5e-245d-4ae5-85df-533468bce888 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:51.491288 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3ef8df5e-245d-4ae5-85df-533468bce888 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29278a807587412b98477436e8312e67'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:51.492987 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:51.502605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3ef8df5e-245d-4ae5-85df-533468bce888 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29278a807587412b98477436e8312e67'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:51.503462 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3ef8df5e-245d-4ae5-85df-533468bce888 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29278a807587412b98477436e8312e67'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:51.513161 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3ef8df5e-245d-4ae5-85df-533468bce888 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29278a807587412b98477436e8312e67'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:51.530143 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3ef8df5e-245d-4ae5-85df-533468bce888 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29278a807587412b98477436e8312e67'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:51.537379 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 993/1985] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:51 2022] DELETE /identity/v3/users/29278a807587412b98477436e8312e67 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:13:51.542590 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c42f30-d062-4679-b355-54170647a019 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:51.543135 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c42f30-d062-4679-b355-54170647a019 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:51.543228 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c42f30-d062-4679-b355-54170647a019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:51.568515 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-32c42f30-d062-4679-b355-54170647a019 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:51.570610 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-32c42f30-d062-4679-b355-54170647a019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:51.646095 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 993/1986] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:51.657521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8df5e-245d-4ae5-85df-533468bce888 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:51.659002 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3ef8df5e-245d-4ae5-85df-533468bce888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:51.724228 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8df5e-245d-4ae5-85df-533468bce888 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:51.727645 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb1e2d-8693-4191-ae59-cdc1cb2a7cc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:51.728268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb1e2d-8693-4191-ae59-cdc1cb2a7cc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:51.728550 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb1e2d-8693-4191-ae59-cdc1cb2a7cc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:51.728827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb1e2d-8693-4191-ae59-cdc1cb2a7cc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:51.738586 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb1e2d-8693-4191-ae59-cdc1cb2a7cc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:51.740354 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afb1e2d-8693-4191-ae59-cdc1cb2a7cc4 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:51.750915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 994/1987] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:52.118291 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c42f30-d062-4679-b355-54170647a019 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:52.120354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-32c42f30-d062-4679-b355-54170647a019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:52.134629 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c42f30-d062-4679-b355-54170647a019 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:52.151114 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfeecec-9e51-4fd5-b88f-747fd870cd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:52.152144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeecec-9e51-4fd5-b88f-747fd870cd44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:52.154281 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeecec-9e51-4fd5-b88f-747fd870cd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:52.154714 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeecec-9e51-4fd5-b88f-747fd870cd44 admin admin] PATH_INFO: `/v3/projects/7e35f509d0cb4826b1639a17018d29a6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:52.156211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfeecec-9e51-4fd5-b88f-747fd870cd44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e35f509d0cb4826b1639a17018d29a6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:52.159387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfeecec-9e51-4fd5-b88f-747fd870cd44 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:52.229881 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:52.233674 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7cfeecec-9e51-4fd5-b88f-747fd870cd44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e35f509d0cb4826b1639a17018d29a6', 'request_id': 'req-7cfeecec-9e51-4fd5-b88f-747fd870cd44'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:52.243577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7cfeecec-9e51-4fd5-b88f-747fd870cd44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e35f509d0cb4826b1639a17018d29a6', 'request_id': 'req-7cfeecec-9e51-4fd5-b88f-747fd870cd44'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:52.251095 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7cfeecec-9e51-4fd5-b88f-747fd870cd44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e35f509d0cb4826b1639a17018d29a6', 'request_id': 'req-7cfeecec-9e51-4fd5-b88f-747fd870cd44'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:52.268467 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7cfeecec-9e51-4fd5-b88f-747fd870cd44 admin admin] The token cache is being invalidate because project 7e35f509d0cb4826b1639a17018d29a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:52.268765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7cfeecec-9e51-4fd5-b88f-747fd870cd44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:52.269715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7cfeecec-9e51-4fd5-b88f-747fd870cd44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:52.271525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 994/1988] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:52 2022] DELETE /identity/v3/projects/7e35f509d0cb4826b1639a17018d29a6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:52.276308 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb1e2d-8693-4191-ae59-cdc1cb2a7cc4 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:52.289152 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb1e2d-8693-4191-ae59-cdc1cb2a7cc4 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:52.356620 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f0242-9cfe-4932-a143-0416e001c0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:52.357385 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f0242-9cfe-4932-a143-0416e001c0b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:52.357681 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f0242-9cfe-4932-a143-0416e001c0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:52.357959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f0242-9cfe-4932-a143-0416e001c0b2 admin admin] PATH_INFO: `/v3/users/fe743b8696814f1da8fb13b4e9f31c82` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:52.359282 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f0242-9cfe-4932-a143-0416e001c0b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe743b8696814f1da8fb13b4e9f31c82)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:52.361742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f0242-9cfe-4932-a143-0416e001c0b2 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:52.434585 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-036f0242-9cfe-4932-a143-0416e001c0b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe743b8696814f1da8fb13b4e9f31c82'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:52.445028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-036f0242-9cfe-4932-a143-0416e001c0b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe743b8696814f1da8fb13b4e9f31c82'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:52.445810 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-036f0242-9cfe-4932-a143-0416e001c0b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe743b8696814f1da8fb13b4e9f31c82'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:52.455325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-036f0242-9cfe-4932-a143-0416e001c0b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe743b8696814f1da8fb13b4e9f31c82'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:52.461959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:52.477614 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-036f0242-9cfe-4932-a143-0416e001c0b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe743b8696814f1da8fb13b4e9f31c82'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:52.484032 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 995/1989] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:52 2022] DELETE /identity/v3/users/fe743b8696814f1da8fb13b4e9f31c82 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:52.844359 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfeecec-9e51-4fd5-b88f-747fd870cd44 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:52.846744 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7cfeecec-9e51-4fd5-b88f-747fd870cd44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:52.885922 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfeecec-9e51-4fd5-b88f-747fd870cd44 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:52.968931 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79be07f-4119-425e-9a4e-d53c9153eebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:52.969631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79be07f-4119-425e-9a4e-d53c9153eebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:52.969934 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79be07f-4119-425e-9a4e-d53c9153eebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:52.970206 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79be07f-4119-425e-9a4e-d53c9153eebf admin admin] PATH_INFO: `/v3/projects/ce68220ead6246258e326e20b93c37ba` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:52.971404 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79be07f-4119-425e-9a4e-d53c9153eebf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce68220ead6246258e326e20b93c37ba)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:52.973438 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79be07f-4119-425e-9a4e-d53c9153eebf admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:53.025228 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c79be07f-4119-425e-9a4e-d53c9153eebf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce68220ead6246258e326e20b93c37ba', 'request_id': 'req-c79be07f-4119-425e-9a4e-d53c9153eebf'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.032034 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c79be07f-4119-425e-9a4e-d53c9153eebf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce68220ead6246258e326e20b93c37ba', 'request_id': 'req-c79be07f-4119-425e-9a4e-d53c9153eebf'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.039515 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c79be07f-4119-425e-9a4e-d53c9153eebf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce68220ead6246258e326e20b93c37ba', 'request_id': 'req-c79be07f-4119-425e-9a4e-d53c9153eebf'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.052217 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c79be07f-4119-425e-9a4e-d53c9153eebf admin admin] The token cache is being invalidate because project ce68220ead6246258e326e20b93c37ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:53.053071 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c79be07f-4119-425e-9a4e-d53c9153eebf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.053948 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c79be07f-4119-425e-9a4e-d53c9153eebf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.055712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 995/1990] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:52 2022] DELETE /identity/v3/projects/ce68220ead6246258e326e20b93c37ba => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:53.059946 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f0242-9cfe-4932-a143-0416e001c0b2 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:53.061457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-036f0242-9cfe-4932-a143-0416e001c0b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:53.075705 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f0242-9cfe-4932-a143-0416e001c0b2 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:53.138359 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09bfb2-4afc-4bca-81a3-ccab50f30ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:53.139059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09bfb2-4afc-4bca-81a3-ccab50f30ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:53.139361 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09bfb2-4afc-4bca-81a3-ccab50f30ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:53.139744 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09bfb2-4afc-4bca-81a3-ccab50f30ef1 admin admin] PATH_INFO: `/v3/users/0d3faa0d52ed4989ab3dbe445231ca48` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:53.140899 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09bfb2-4afc-4bca-81a3-ccab50f30ef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d3faa0d52ed4989ab3dbe445231ca48)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:53.143016 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09bfb2-4afc-4bca-81a3-ccab50f30ef1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:53.198879 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9c09bfb2-4afc-4bca-81a3-ccab50f30ef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d3faa0d52ed4989ab3dbe445231ca48'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.208712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9c09bfb2-4afc-4bca-81a3-ccab50f30ef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d3faa0d52ed4989ab3dbe445231ca48'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.209553 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9c09bfb2-4afc-4bca-81a3-ccab50f30ef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d3faa0d52ed4989ab3dbe445231ca48'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.218188 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9c09bfb2-4afc-4bca-81a3-ccab50f30ef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d3faa0d52ed4989ab3dbe445231ca48'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.231952 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9c09bfb2-4afc-4bca-81a3-ccab50f30ef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d3faa0d52ed4989ab3dbe445231ca48'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.238463 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 996/1991] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:53 2022] DELETE /identity/v3/users/0d3faa0d52ed4989ab3dbe445231ca48 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:53.266131 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79be07f-4119-425e-9a4e-d53c9153eebf admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:53.267910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c79be07f-4119-425e-9a4e-d53c9153eebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:53.288704 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79be07f-4119-425e-9a4e-d53c9153eebf admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:53.360300 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaaa87-8795-419a-aeb6-b41dc6426697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:53.361066 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaaa87-8795-419a-aeb6-b41dc6426697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:53.361482 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaaa87-8795-419a-aeb6-b41dc6426697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:53.361819 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aaaa87-8795-419a-aeb6-b41dc6426697 admin admin] PATH_INFO: `/v3/users/7218a9eec3d8414d86bd505bd2d06425` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:53.363155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaaa87-8795-419a-aeb6-b41dc6426697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7218a9eec3d8414d86bd505bd2d06425)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:53.365877 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aaaa87-8795-419a-aeb6-b41dc6426697 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:53.372846 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:53.445673 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-19aaaa87-8795-419a-aeb6-b41dc6426697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7218a9eec3d8414d86bd505bd2d06425'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.454533 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-19aaaa87-8795-419a-aeb6-b41dc6426697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7218a9eec3d8414d86bd505bd2d06425'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.455270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-19aaaa87-8795-419a-aeb6-b41dc6426697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7218a9eec3d8414d86bd505bd2d06425'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.465995 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-19aaaa87-8795-419a-aeb6-b41dc6426697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7218a9eec3d8414d86bd505bd2d06425'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.482741 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-19aaaa87-8795-419a-aeb6-b41dc6426697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7218a9eec3d8414d86bd505bd2d06425'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.490177 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 996/1992] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:13:53 2022] DELETE /identity/v3/users/7218a9eec3d8414d86bd505bd2d06425 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:53.494887 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6046d-8f3c-48f4-8386-c9e85d4c2b2c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:53.495364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6046d-8f3c-48f4-8386-c9e85d4c2b2c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:53.495726 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6046d-8f3c-48f4-8386-c9e85d4c2b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:53.519454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-29e6046d-8f3c-48f4-8386-c9e85d4c2b2c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:53.520680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-29e6046d-8f3c-48f4-8386-c9e85d4c2b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:53.570052 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaaa87-8795-419a-aeb6-b41dc6426697 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:53.571750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-19aaaa87-8795-419a-aeb6-b41dc6426697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:53.620795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 997/1993] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:53.638919 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6046d-8f3c-48f4-8386-c9e85d4c2b2c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:53.640467 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-29e6046d-8f3c-48f4-8386-c9e85d4c2b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:53.650251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:53.700019 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aaaa87-8795-419a-aeb6-b41dc6426697 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:53.706697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703c7b5-38c5-412a-9271-1e9b98f25edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:53.708057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703c7b5-38c5-412a-9271-1e9b98f25edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:53.708441 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703c7b5-38c5-412a-9271-1e9b98f25edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:53.708888 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703c7b5-38c5-412a-9271-1e9b98f25edb admin admin] PATH_INFO: `/v3/projects/d557f4d5eb8a4d958fc196a1008e9da0` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:53.710262 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703c7b5-38c5-412a-9271-1e9b98f25edb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d557f4d5eb8a4d958fc196a1008e9da0)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:53.712217 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703c7b5-38c5-412a-9271-1e9b98f25edb admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:53.732966 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6046d-8f3c-48f4-8386-c9e85d4c2b2c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:53.737029 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398dd5d-d121-4972-a2f3-2045ac866f71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:53.738233 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398dd5d-d121-4972-a2f3-2045ac866f71 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:53.738627 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398dd5d-d121-4972-a2f3-2045ac866f71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:53.738954 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398dd5d-d121-4972-a2f3-2045ac866f71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:53.757158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398dd5d-d121-4972-a2f3-2045ac866f71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:53.758943 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d398dd5d-d121-4972-a2f3-2045ac866f71 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:53.783709 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 998/1994] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:13:53.801601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e703c7b5-38c5-412a-9271-1e9b98f25edb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd557f4d5eb8a4d958fc196a1008e9da0', 'request_id': 'req-e703c7b5-38c5-412a-9271-1e9b98f25edb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.815594 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e703c7b5-38c5-412a-9271-1e9b98f25edb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd557f4d5eb8a4d958fc196a1008e9da0', 'request_id': 'req-e703c7b5-38c5-412a-9271-1e9b98f25edb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.828624 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e703c7b5-38c5-412a-9271-1e9b98f25edb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd557f4d5eb8a4d958fc196a1008e9da0', 'request_id': 'req-e703c7b5-38c5-412a-9271-1e9b98f25edb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.847321 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e703c7b5-38c5-412a-9271-1e9b98f25edb admin admin] The token cache is being invalidate because project d557f4d5eb8a4d958fc196a1008e9da0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:53.848746 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e703c7b5-38c5-412a-9271-1e9b98f25edb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.850793 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e703c7b5-38c5-412a-9271-1e9b98f25edb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:53.862276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 997/1995] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:53 2022] DELETE /identity/v3/projects/d557f4d5eb8a4d958fc196a1008e9da0 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:13:54.253496 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398dd5d-d121-4972-a2f3-2045ac866f71 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:54.255313 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d398dd5d-d121-4972-a2f3-2045ac866f71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:54.288042 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398dd5d-d121-4972-a2f3-2045ac866f71 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:54.363190 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13961b0a-b594-43a1-8c88-d3f1ae6ca3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:54.363843 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13961b0a-b594-43a1-8c88-d3f1ae6ca3b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:54.364129 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13961b0a-b594-43a1-8c88-d3f1ae6ca3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:54.364401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13961b0a-b594-43a1-8c88-d3f1ae6ca3b6 admin admin] PATH_INFO: `/v3/projects/2c415e743af644fc834efc9ad512898a` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:54.366350 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13961b0a-b594-43a1-8c88-d3f1ae6ca3b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c415e743af644fc834efc9ad512898a)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:54.368567 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13961b0a-b594-43a1-8c88-d3f1ae6ca3b6 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:54.425837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-13961b0a-b594-43a1-8c88-d3f1ae6ca3b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c415e743af644fc834efc9ad512898a', 'request_id': 'req-13961b0a-b594-43a1-8c88-d3f1ae6ca3b6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:54.432004 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-13961b0a-b594-43a1-8c88-d3f1ae6ca3b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c415e743af644fc834efc9ad512898a', 'request_id': 'req-13961b0a-b594-43a1-8c88-d3f1ae6ca3b6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:54.438507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-13961b0a-b594-43a1-8c88-d3f1ae6ca3b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c415e743af644fc834efc9ad512898a', 'request_id': 'req-13961b0a-b594-43a1-8c88-d3f1ae6ca3b6'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:54.451838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-13961b0a-b594-43a1-8c88-d3f1ae6ca3b6 admin admin] The token cache is being invalidate because project 2c415e743af644fc834efc9ad512898a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:13:54.452126 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-13961b0a-b594-43a1-8c88-d3f1ae6ca3b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:54.452781 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-13961b0a-b594-43a1-8c88-d3f1ae6ca3b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:13:54.454271 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 999/1996] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:13:54 2022] DELETE /identity/v3/projects/2c415e743af644fc834efc9ad512898a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:54.459303 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c7ff0-7c18-475f-a8bc-d7ff4a9a9301 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:54.459602 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c7ff0-7c18-475f-a8bc-d7ff4a9a9301 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:54.459905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c7ff0-7c18-475f-a8bc-d7ff4a9a9301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:54.480395 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9c3c7ff0-7c18-475f-a8bc-d7ff4a9a9301 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:54.481648 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9c3c7ff0-7c18-475f-a8bc-d7ff4a9a9301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:54.492272 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:54.543468 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 998/1997] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:54.551288 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a769e4e-e57b-4515-8cbc-b9928f005090 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:54.551821 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a769e4e-e57b-4515-8cbc-b9928f005090 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:54.551915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a769e4e-e57b-4515-8cbc-b9928f005090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:54.572556 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1a769e4e-e57b-4515-8cbc-b9928f005090 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:54.573758 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1a769e4e-e57b-4515-8cbc-b9928f005090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:54.581574 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1000/1998] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:54.585734 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c7ff0-7c18-475f-a8bc-d7ff4a9a9301 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:54.595338 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c7ff0-7c18-475f-a8bc-d7ff4a9a9301 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:54.600644 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866efe9a-54e4-49e3-9c7c-f1ff2f5cf52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:54.601252 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866efe9a-54e4-49e3-9c7c-f1ff2f5cf52f admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:54.601535 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866efe9a-54e4-49e3-9c7c-f1ff2f5cf52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:54.601811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866efe9a-54e4-49e3-9c7c-f1ff2f5cf52f admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:54.602457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866efe9a-54e4-49e3-9c7c-f1ff2f5cf52f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:13:54.602762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866efe9a-54e4-49e3-9c7c-f1ff2f5cf52f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:54.604387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866efe9a-54e4-49e3-9c7c-f1ff2f5cf52f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:54.616641 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 999/1999] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:13:54.620826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a769e4e-e57b-4515-8cbc-b9928f005090 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:54.629943 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a769e4e-e57b-4515-8cbc-b9928f005090 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:54.634636 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfdffa-54aa-45ec-9e0c-041247ca1ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:54.635268 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfdffa-54aa-45ec-9e0c-041247ca1ddf admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:54.635552 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfdffa-54aa-45ec-9e0c-041247ca1ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:54.635825 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfdffa-54aa-45ec-9e0c-041247ca1ddf admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:54.636499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfdffa-54aa-45ec-9e0c-041247ca1ddf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:54.638399 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfdffa-54aa-45ec-9e0c-041247ca1ddf admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:54.666262 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1001/2000] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:13:54 2022] POST /identity/v3/projects => generated 397 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:54.670356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866efe9a-54e4-49e3-9c7c-f1ff2f5cf52f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:54.681329 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866efe9a-54e4-49e3-9c7c-f1ff2f5cf52f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:54.740667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507df8b-b944-4484-a134-6ad5ca84aea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:54.741417 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507df8b-b944-4484-a134-6ad5ca84aea0 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:54.741738 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507df8b-b944-4484-a134-6ad5ca84aea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:54.742076 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507df8b-b944-4484-a134-6ad5ca84aea0 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:54.742765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507df8b-b944-4484-a134-6ad5ca84aea0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:54.744944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507df8b-b944-4484-a134-6ad5ca84aea0 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:54.767817 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1000/2001] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:13:54 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:13:54.772806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfdffa-54aa-45ec-9e0c-041247ca1ddf admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:54.783901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfdffa-54aa-45ec-9e0c-041247ca1ddf admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:54.788920 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a886fd9-c25b-4996-b80c-22590a7a649b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:54.789535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a886fd9-c25b-4996-b80c-22590a7a649b admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:54.789817 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a886fd9-c25b-4996-b80c-22590a7a649b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:54.790091 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a886fd9-c25b-4996-b80c-22590a7a649b admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:54.790744 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a886fd9-c25b-4996-b80c-22590a7a649b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:54.792583 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a886fd9-c25b-4996-b80c-22590a7a649b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:54.794753 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:54.804316 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1002/2002] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:54.807913 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507df8b-b944-4484-a134-6ad5ca84aea0 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:54.819598 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507df8b-b944-4484-a134-6ad5ca84aea0 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:54.824589 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88fd9f-c407-48e9-8f19-76fad70deac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:54.825270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88fd9f-c407-48e9-8f19-76fad70deac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:54.825623 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88fd9f-c407-48e9-8f19-76fad70deac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:54.825898 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88fd9f-c407-48e9-8f19-76fad70deac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:54.826569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88fd9f-c407-48e9-8f19-76fad70deac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:54.828407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88fd9f-c407-48e9-8f19-76fad70deac1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:54.840379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1001/2003] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:13:54.844614 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a886fd9-c25b-4996-b80c-22590a7a649b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:54.855610 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a886fd9-c25b-4996-b80c-22590a7a649b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:54.860198 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de40e0a-4798-4c5a-8c53-2bb64057efba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:54.860840 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de40e0a-4798-4c5a-8c53-2bb64057efba admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:54.861120 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de40e0a-4798-4c5a-8c53-2bb64057efba admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:54.861415 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de40e0a-4798-4c5a-8c53-2bb64057efba admin admin] PATH_INFO: `/v3/projects/2a7165949ec540cab71d169a743762c2/users/d1293478dbf74caea91becb79ce94c32/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:54.876940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de40e0a-4798-4c5a-8c53-2bb64057efba admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7165949ec540cab71d169a743762c2, user_id=d1293478dbf74caea91becb79ce94c32, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:54.879760 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de40e0a-4798-4c5a-8c53-2bb64057efba admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:54.894171 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1003/2004] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:13:54 2022] PUT /identity/v3/projects/2a7165949ec540cab71d169a743762c2/users/d1293478dbf74caea91becb79ce94c32/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:13:54.905006 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fcc75-1823-4d40-8453-0433107f6ddc None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:54.905506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fcc75-1823-4d40-8453-0433107f6ddc None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:54.905614 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fcc75-1823-4d40-8453-0433107f6ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:54.933976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-774fcc75-1823-4d40-8453-0433107f6ddc None None] MFA Rules not processed for user `d1293478dbf74caea91becb79ce94c32`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:54.935080 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-774fcc75-1823-4d40-8453-0433107f6ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:55.004993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1002/2005] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:13:54 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:55.019470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de40e0a-4798-4c5a-8c53-2bb64057efba admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:55.021218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7de40e0a-4798-4c5a-8c53-2bb64057efba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:55.080643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de40e0a-4798-4c5a-8c53-2bb64057efba admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:55.084350 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f61ec8-373e-4105-9d5c-d85872b74bbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:55.084961 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f61ec8-373e-4105-9d5c-d85872b74bbd service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:55.085235 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f61ec8-373e-4105-9d5c-d85872b74bbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:55.085506 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f61ec8-373e-4105-9d5c-d85872b74bbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:55.094141 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f61ec8-373e-4105-9d5c-d85872b74bbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:55.096217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f61ec8-373e-4105-9d5c-d85872b74bbd service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:55.104646 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1004/2006] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:13:55 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:13:58.203009 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd16d5-c742-4762-a425-758a3e693c19 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:58.203501 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd16d5-c742-4762-a425-758a3e693c19 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:58.203596 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd16d5-c742-4762-a425-758a3e693c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:58.225786 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e5cd16d5-c742-4762-a425-758a3e693c19 None None] MFA Rules not processed for user `280d47ed79f14105a83b18b2cac58b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:13:58.227064 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e5cd16d5-c742-4762-a425-758a3e693c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:58.232456 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:13:58.285329 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1003/2007] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Oct 17 03:13:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:13:58.340047 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f61ec8-373e-4105-9d5c-d85872b74bbd service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:13:58.342004 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-53f61ec8-373e-4105-9d5c-d85872b74bbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:13:58.416809 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f61ec8-373e-4105-9d5c-d85872b74bbd service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:13:58.421791 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538a48d-e4f1-4892-a16b-07dd8e262de4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eceda8b3a8ba4233851d0138d8f4d06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:13:58.422435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538a48d-e4f1-4892-a16b-07dd8e262de4 service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:13:58.422720 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538a48d-e4f1-4892-a16b-07dd8e262de4 service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:13:58.423040 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538a48d-e4f1-4892-a16b-07dd8e262de4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:13:58.431921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538a48d-e4f1-4892-a16b-07dd8e262de4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:13:58.433513 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1538a48d-e4f1-4892-a16b-07dd8e262de4 service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:13:58.441577 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1005/2008] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Oct 17 03:13:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:14:04.684485 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd16d5-c742-4762-a425-758a3e693c19 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:04.762757 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd16d5-c742-4762-a425-758a3e693c19 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:04.768700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71103400-2652-4607-9c68-31708f84ce14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:04.772085 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71103400-2652-4607-9c68-31708f84ce14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:04.772085 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71103400-2652-4607-9c68-31708f84ce14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:04.772085 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71103400-2652-4607-9c68-31708f84ce14 admin admin] PATH_INFO: `/v3/users/d1293478dbf74caea91becb79ce94c32` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:04.772085 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71103400-2652-4607-9c68-31708f84ce14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1293478dbf74caea91becb79ce94c32)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:04.773700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71103400-2652-4607-9c68-31708f84ce14 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:04.851909 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-71103400-2652-4607-9c68-31708f84ce14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1293478dbf74caea91becb79ce94c32'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:04.867538 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-71103400-2652-4607-9c68-31708f84ce14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1293478dbf74caea91becb79ce94c32'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:04.868823 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-71103400-2652-4607-9c68-31708f84ce14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1293478dbf74caea91becb79ce94c32'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:04.882494 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-71103400-2652-4607-9c68-31708f84ce14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1293478dbf74caea91becb79ce94c32'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:04.902226 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-71103400-2652-4607-9c68-31708f84ce14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1293478dbf74caea91becb79ce94c32'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:04.914929 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1004/2009] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:14:04 2022] DELETE /identity/v3/users/d1293478dbf74caea91becb79ce94c32 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:14:04.917030 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1759e-fe9c-4108-9fa1-cce46d022515 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:04.917916 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1759e-fe9c-4108-9fa1-cce46d022515 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:04.918179 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1759e-fe9c-4108-9fa1-cce46d022515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:04.940969 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-eab1759e-fe9c-4108-9fa1-cce46d022515 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:14:04.943614 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-eab1759e-fe9c-4108-9fa1-cce46d022515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:05.030637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1006/2010] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:14:05.041908 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71103400-2652-4607-9c68-31708f84ce14 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:05.043708 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-71103400-2652-4607-9c68-31708f84ce14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:05.089092 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:14:05.112927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71103400-2652-4607-9c68-31708f84ce14 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:05.117222 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8c8ee-864a-40bc-ae0a-da98bc6b9b4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:05.117971 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8c8ee-864a-40bc-ae0a-da98bc6b9b4c service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:05.118254 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8c8ee-864a-40bc-ae0a-da98bc6b9b4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:05.118530 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8c8ee-864a-40bc-ae0a-da98bc6b9b4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:05.129240 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8c8ee-864a-40bc-ae0a-da98bc6b9b4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:05.131807 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8c8ee-864a-40bc-ae0a-da98bc6b9b4c service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:05.142255 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1005/2011] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:14:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:14:05.469381 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1759e-fe9c-4108-9fa1-cce46d022515 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:05.471148 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-eab1759e-fe9c-4108-9fa1-cce46d022515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:05.486343 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1759e-fe9c-4108-9fa1-cce46d022515 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:05.492747 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae8934-0462-47d3-a006-99771efe1768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:05.493687 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae8934-0462-47d3-a006-99771efe1768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:05.494238 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae8934-0462-47d3-a006-99771efe1768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:05.494700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ae8934-0462-47d3-a006-99771efe1768 admin admin] PATH_INFO: `/v3/projects/2a7165949ec540cab71d169a743762c2` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:05.496448 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae8934-0462-47d3-a006-99771efe1768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a7165949ec540cab71d169a743762c2)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:05.498911 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ae8934-0462-47d3-a006-99771efe1768 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:05.501083 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:14:05.627316 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d5ae8934-0462-47d3-a006-99771efe1768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a7165949ec540cab71d169a743762c2', 'request_id': 'req-d5ae8934-0462-47d3-a006-99771efe1768'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:05.651008 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d5ae8934-0462-47d3-a006-99771efe1768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a7165949ec540cab71d169a743762c2', 'request_id': 'req-d5ae8934-0462-47d3-a006-99771efe1768'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:05.667085 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d5ae8934-0462-47d3-a006-99771efe1768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a7165949ec540cab71d169a743762c2', 'request_id': 'req-d5ae8934-0462-47d3-a006-99771efe1768'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:05.697882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d5ae8934-0462-47d3-a006-99771efe1768 admin admin] The token cache is being invalidate because project 2a7165949ec540cab71d169a743762c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:14:05.697882 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d5ae8934-0462-47d3-a006-99771efe1768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:05.698670 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-d5ae8934-0462-47d3-a006-99771efe1768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:05.700723 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1007/2012] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:14:05 2022] DELETE /identity/v3/projects/2a7165949ec540cab71d169a743762c2 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:14:07.830051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a9bde-d34e-4c36-8a81-041dea8ca048 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:07.830599 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a9bde-d34e-4c36-8a81-041dea8ca048 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:07.830721 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a9bde-d34e-4c36-8a81-041dea8ca048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:07.854551 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ca3a9bde-d34e-4c36-8a81-041dea8ca048 None None] MFA Rules not processed for user `280d47ed79f14105a83b18b2cac58b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:14:07.855814 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ca3a9bde-d34e-4c36-8a81-041dea8ca048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:07.930260 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1006/2013] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Oct 17 03:14:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:14:07.946441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae8934-0462-47d3-a006-99771efe1768 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:07.947819 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d5ae8934-0462-47d3-a006-99771efe1768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:08.068222 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ae8934-0462-47d3-a006-99771efe1768 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:08.077333 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef1b09-1c74-400b-a096-0d635775d76c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eceda8b3a8ba4233851d0138d8f4d06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:08.077994 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef1b09-1c74-400b-a096-0d635775d76c service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:08.078325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef1b09-1c74-400b-a096-0d635775d76c service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:08.078746 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef1b09-1c74-400b-a096-0d635775d76c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:08.104892 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef1b09-1c74-400b-a096-0d635775d76c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:08.106540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef1b09-1c74-400b-a096-0d635775d76c service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:08.116660 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1008/2014] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Oct 17 03:14:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:14:11.901234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67be42-9570-4ccc-a9b9-c710edde641a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:11.901750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67be42-9570-4ccc-a9b9-c710edde641a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:11.902084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67be42-9570-4ccc-a9b9-c710edde641a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:11.940435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-5a67be42-9570-4ccc-a9b9-c710edde641a None None] MFA Rules not processed for user `9c1787a9ef7c44e28076aaa80c0152fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:14:11.941877 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5a67be42-9570-4ccc-a9b9-c710edde641a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:12.116492 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1007/2015] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:14:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:14:12.641188 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b667b-1b1e-46ed-adbf-2324505f0f27 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:12.641929 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b667b-1b1e-46ed-adbf-2324505f0f27 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:12.641929 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b667b-1b1e-46ed-adbf-2324505f0f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:12.691446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-7f0b667b-1b1e-46ed-adbf-2324505f0f27 None None] MFA Rules not processed for user `1c786544e8fb4e8e8c3a4990a0f36725`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:14:12.695653 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7f0b667b-1b1e-46ed-adbf-2324505f0f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:12.816873 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1009/2016] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:14:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:14:20.638754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480491d-b22e-4b5c-a9c0-e76b833a02fc None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:20.639332 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480491d-b22e-4b5c-a9c0-e76b833a02fc None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:20.639332 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480491d-b22e-4b5c-a9c0-e76b833a02fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:20.658846 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9480491d-b22e-4b5c-a9c0-e76b833a02fc None None] MFA Rules not processed for user `280d47ed79f14105a83b18b2cac58b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:14:20.660065 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9480491d-b22e-4b5c-a9c0-e76b833a02fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:20.667857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1008/2017] 10.222.0.27 () {58 vars in 967 bytes} [Mon Oct 17 03:14:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:14:20.680504 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b667b-1b1e-46ed-adbf-2324505f0f27 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:20.692217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b667b-1b1e-46ed-adbf-2324505f0f27 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:20.698004 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15441a38-147c-4896-b5ba-5c898322facd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eceda8b3a8ba4233851d0138d8f4d06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:20.698614 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15441a38-147c-4896-b5ba-5c898322facd service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:20.698918 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15441a38-147c-4896-b5ba-5c898322facd service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:20.699193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15441a38-147c-4896-b5ba-5c898322facd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:20.707340 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15441a38-147c-4896-b5ba-5c898322facd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:20.709556 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15441a38-147c-4896-b5ba-5c898322facd service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:20.717375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1010/2018] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Oct 17 03:14:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 17 03:14:25.568710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480491d-b22e-4b5c-a9c0-e76b833a02fc None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:25.571019 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9480491d-b22e-4b5c-a9c0-e76b833a02fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:25.666812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480491d-b22e-4b5c-a9c0-e76b833a02fc None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:25.675136 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3bfc1-05a8-4b10-be04-9aac650bcc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:25.676136 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3bfc1-05a8-4b10-be04-9aac650bcc21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:25.676489 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3bfc1-05a8-4b10-be04-9aac650bcc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:25.676771 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3bfc1-05a8-4b10-be04-9aac650bcc21 admin admin] PATH_INFO: `/v3/users/c58fbb2d79954020aa1328ef6bf925e4` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:25.678537 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3bfc1-05a8-4b10-be04-9aac650bcc21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c58fbb2d79954020aa1328ef6bf925e4)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:25.680751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3bfc1-05a8-4b10-be04-9aac650bcc21 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:25.709343 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:14:25.737227 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-50e3bfc1-05a8-4b10-be04-9aac650bcc21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c58fbb2d79954020aa1328ef6bf925e4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:25.744622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-50e3bfc1-05a8-4b10-be04-9aac650bcc21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c58fbb2d79954020aa1328ef6bf925e4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:25.745279 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-50e3bfc1-05a8-4b10-be04-9aac650bcc21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c58fbb2d79954020aa1328ef6bf925e4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:25.752356 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-50e3bfc1-05a8-4b10-be04-9aac650bcc21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c58fbb2d79954020aa1328ef6bf925e4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:25.765265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-50e3bfc1-05a8-4b10-be04-9aac650bcc21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c58fbb2d79954020aa1328ef6bf925e4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:25.771295 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1009/2019] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:14:25 2022] DELETE /identity/v3/users/c58fbb2d79954020aa1328ef6bf925e4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:14:26.018854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15441a38-147c-4896-b5ba-5c898322facd service swift] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:26.030196 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15441a38-147c-4896-b5ba-5c898322facd service swift] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:26.039067 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:14:26.093983 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8047e-6485-4b4c-8fad-65fbc4571d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:26.094634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8047e-6485-4b4c-8fad-65fbc4571d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:26.094951 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8047e-6485-4b4c-8fad-65fbc4571d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:26.095227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8047e-6485-4b4c-8fad-65fbc4571d82 admin admin] PATH_INFO: `/v3/projects/b87c159313f74c77a95172124e82dc25` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:26.096347 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8047e-6485-4b4c-8fad-65fbc4571d82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b87c159313f74c77a95172124e82dc25)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:26.098251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8047e-6485-4b4c-8fad-65fbc4571d82 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:26.144018 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-91e8047e-6485-4b4c-8fad-65fbc4571d82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b87c159313f74c77a95172124e82dc25', 'request_id': 'req-91e8047e-6485-4b4c-8fad-65fbc4571d82'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:26.149647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-91e8047e-6485-4b4c-8fad-65fbc4571d82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b87c159313f74c77a95172124e82dc25', 'request_id': 'req-91e8047e-6485-4b4c-8fad-65fbc4571d82'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:26.156006 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-91e8047e-6485-4b4c-8fad-65fbc4571d82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b87c159313f74c77a95172124e82dc25', 'request_id': 'req-91e8047e-6485-4b4c-8fad-65fbc4571d82'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:26.168571 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-91e8047e-6485-4b4c-8fad-65fbc4571d82 admin admin] The token cache is being invalidate because project b87c159313f74c77a95172124e82dc25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:14:26.168858 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-91e8047e-6485-4b4c-8fad-65fbc4571d82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:26.169467 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-91e8047e-6485-4b4c-8fad-65fbc4571d82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:14:26.171072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1011/2020] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:14:26 2022] DELETE /identity/v3/projects/b87c159313f74c77a95172124e82dc25 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:14:26.176092 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11276bb4-c3be-4cd3-9991-cbba39185943 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:26.176404 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11276bb4-c3be-4cd3-9991-cbba39185943 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:26.176676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11276bb4-c3be-4cd3-9991-cbba39185943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:26.197428 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-11276bb4-c3be-4cd3-9991-cbba39185943 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:14:26.198622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-11276bb4-c3be-4cd3-9991-cbba39185943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:26.257557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1010/2021] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:14:26.265153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e17c3-65c7-4dbc-bc7c-a0c5d51840bd None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:26.265490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e17c3-65c7-4dbc-bc7c-a0c5d51840bd None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:26.265770 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e17c3-65c7-4dbc-bc7c-a0c5d51840bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:26.286226 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-9b9e17c3-65c7-4dbc-bc7c-a0c5d51840bd None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:14:26.288620 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9b9e17c3-65c7-4dbc-bc7c-a0c5d51840bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:26.296457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1012/2022] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:14:26.300589 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11276bb4-c3be-4cd3-9991-cbba39185943 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:26.311334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11276bb4-c3be-4cd3-9991-cbba39185943 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:26.316096 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b5a99-b4ec-4ef7-a1b7-2dd51d833982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:26.316691 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b5a99-b4ec-4ef7-a1b7-2dd51d833982 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:26.316969 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b5a99-b4ec-4ef7-a1b7-2dd51d833982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:26.317238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b5a99-b4ec-4ef7-a1b7-2dd51d833982 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:26.317888 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b5a99-b4ec-4ef7-a1b7-2dd51d833982 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:14:26.318242 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b5a99-b4ec-4ef7-a1b7-2dd51d833982 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:26.319871 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b5a99-b4ec-4ef7-a1b7-2dd51d833982 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:26.331299 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1011/2023] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:14:26.337237 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e17c3-65c7-4dbc-bc7c-a0c5d51840bd None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:26.348133 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e17c3-65c7-4dbc-bc7c-a0c5d51840bd None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:26.353514 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9627b7a-f200-4cb1-b729-8e625ff5c9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:26.354109 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9627b7a-f200-4cb1-b729-8e625ff5c9be admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:26.354385 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9627b7a-f200-4cb1-b729-8e625ff5c9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:26.354680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9627b7a-f200-4cb1-b729-8e625ff5c9be admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:26.355344 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9627b7a-f200-4cb1-b729-8e625ff5c9be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:26.357242 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9627b7a-f200-4cb1-b729-8e625ff5c9be admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:26.384747 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1013/2024] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:14:26 2022] POST /identity/v3/projects => generated 361 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:14:26.388850 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b5a99-b4ec-4ef7-a1b7-2dd51d833982 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:26.399515 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b5a99-b4ec-4ef7-a1b7-2dd51d833982 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:26.458842 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217eb415-879c-4592-9994-8f8c8b5c953e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:26.459719 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217eb415-879c-4592-9994-8f8c8b5c953e admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:26.459998 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217eb415-879c-4592-9994-8f8c8b5c953e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:26.460273 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217eb415-879c-4592-9994-8f8c8b5c953e admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:26.460945 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217eb415-879c-4592-9994-8f8c8b5c953e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:26.462905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217eb415-879c-4592-9994-8f8c8b5c953e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:26.488167 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1012/2025] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:14:26 2022] POST /identity/v3/users => generated 281 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:14:26.489512 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9627b7a-f200-4cb1-b729-8e625ff5c9be admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:26.500239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9627b7a-f200-4cb1-b729-8e625ff5c9be admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:26.504939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d07c90-db1a-45e8-9827-afe3c3b9b944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:26.505675 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d07c90-db1a-45e8-9827-afe3c3b9b944 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:26.506077 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d07c90-db1a-45e8-9827-afe3c3b9b944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:26.506350 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d07c90-db1a-45e8-9827-afe3c3b9b944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:26.507061 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d07c90-db1a-45e8-9827-afe3c3b9b944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:26.508668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d07c90-db1a-45e8-9827-afe3c3b9b944 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:26.523529 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1014/2026] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:14:26.524858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217eb415-879c-4592-9994-8f8c8b5c953e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:26.535042 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217eb415-879c-4592-9994-8f8c8b5c953e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:26.540667 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d470b0-f0ce-4fca-8b47-2097a7629d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:26.541275 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d470b0-f0ce-4fca-8b47-2097a7629d31 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:26.541555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d470b0-f0ce-4fca-8b47-2097a7629d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:26.541828 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d470b0-f0ce-4fca-8b47-2097a7629d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:26.542474 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d470b0-f0ce-4fca-8b47-2097a7629d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:26.544102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d470b0-f0ce-4fca-8b47-2097a7629d31 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:26.555124 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1013/2027] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:14:26.559369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d07c90-db1a-45e8-9827-afe3c3b9b944 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:26.571343 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d07c90-db1a-45e8-9827-afe3c3b9b944 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:26.576702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327112d9-8e09-4554-aa04-d171bb01b2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:26.577389 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327112d9-8e09-4554-aa04-d171bb01b2f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:26.577710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327112d9-8e09-4554-aa04-d171bb01b2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:26.577964 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327112d9-8e09-4554-aa04-d171bb01b2f8 admin admin] PATH_INFO: `/v3/projects/862ff19a5e444868be6cc4cfa3a11cfb/users/8229b889af004764b026600e9fb9b6b9/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:26.592848 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327112d9-8e09-4554-aa04-d171bb01b2f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=862ff19a5e444868be6cc4cfa3a11cfb, user_id=8229b889af004764b026600e9fb9b6b9, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:26.595549 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327112d9-8e09-4554-aa04-d171bb01b2f8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:26.608537 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1015/2028] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:14:26 2022] PUT /identity/v3/projects/862ff19a5e444868be6cc4cfa3a11cfb/users/8229b889af004764b026600e9fb9b6b9/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:14:26.619749 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45fe5d-6545-4f58-be99-b7ed91d22c47 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:26.620237 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45fe5d-6545-4f58-be99-b7ed91d22c47 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:26.620361 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45fe5d-6545-4f58-be99-b7ed91d22c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:26.633455 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:14:26.649561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-df45fe5d-6545-4f58-be99-b7ed91d22c47 None None] MFA Rules not processed for user `8229b889af004764b026600e9fb9b6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:14:26.650803 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-df45fe5d-6545-4f58-be99-b7ed91d22c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:26.714926 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1014/2029] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:14:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:14:26.725969 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327112d9-8e09-4554-aa04-d171bb01b2f8 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:26.727767 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-327112d9-8e09-4554-aa04-d171bb01b2f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:26.770647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:14:26.788228 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327112d9-8e09-4554-aa04-d171bb01b2f8 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:26.791949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043ff8f-0488-43fa-ac2a-c60f8e553775 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:26.792557 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043ff8f-0488-43fa-ac2a-c60f8e553775 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:26.792840 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043ff8f-0488-43fa-ac2a-c60f8e553775 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:26.793112 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043ff8f-0488-43fa-ac2a-c60f8e553775 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:26.801850 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043ff8f-0488-43fa-ac2a-c60f8e553775 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:26.803347 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043ff8f-0488-43fa-ac2a-c60f8e553775 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:26.812454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1016/2030] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:14:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:14:32.061811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce46a9-fa5f-4fa7-a05e-0e00ca27cb16 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:32.062336 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce46a9-fa5f-4fa7-a05e-0e00ca27cb16 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:32.062594 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce46a9-fa5f-4fa7-a05e-0e00ca27cb16 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:32.065140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1015/2031] 10.222.0.27 () {54 vars in 921 bytes} [Mon Oct 17 03:14:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:14:32.073414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a0d7b-ed2f-43b8-8990-5a541f8b3130 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:32.073975 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a0d7b-ed2f-43b8-8990-5a541f8b3130 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:32.073975 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a0d7b-ed2f-43b8-8990-5a541f8b3130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:32.099426 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8d0a0d7b-ed2f-43b8-8990-5a541f8b3130 None None] MFA Rules not processed for user `01637fc07ba846aebcb42325bed46216`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:14:32.100641 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8d0a0d7b-ed2f-43b8-8990-5a541f8b3130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:32.193233 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1017/2032] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Oct 17 03:14:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:14:32.202537 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce46a9-fa5f-4fa7-a05e-0e00ca27cb16 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:32.205843 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a6ce46a9-fa5f-4fa7-a05e-0e00ca27cb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:32.357716 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce46a9-fa5f-4fa7-a05e-0e00ca27cb16 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:32.362365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2cc452-9d80-4aec-aeac-2be2ccdf03b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8aa29639a3143d2a270e3248d2205d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:32.363446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cc452-9d80-4aec-aeac-2be2ccdf03b8 service glance] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:32.363728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cc452-9d80-4aec-aeac-2be2ccdf03b8 service glance] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:32.363996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cc452-9d80-4aec-aeac-2be2ccdf03b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:32.375317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2cc452-9d80-4aec-aeac-2be2ccdf03b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:32.376753 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2cc452-9d80-4aec-aeac-2be2ccdf03b8 service glance] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:32.388176 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1016/2033] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:14:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:14:32.493571 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e0170-3ebc-46c3-a50e-ede9fc6f250a None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:32.493877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e0170-3ebc-46c3-a50e-ede9fc6f250a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:32.494154 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e0170-3ebc-46c3-a50e-ede9fc6f250a None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:32.495894 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1018/2034] 10.222.0.27 () {54 vars in 921 bytes} [Mon Oct 17 03:14:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:14:32.501581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b423b-9237-45e9-91d5-547607672d90 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:32.501983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b423b-9237-45e9-91d5-547607672d90 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:32.502735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b423b-9237-45e9-91d5-547607672d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:32.539645 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a32b423b-9237-45e9-91d5-547607672d90 None None] MFA Rules not processed for user `526933e6ad004493b071f5da5caa09a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:14:32.540783 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a32b423b-9237-45e9-91d5-547607672d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:32.617531 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1017/2035] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Oct 17 03:14:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:14:32.623498 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e0170-3ebc-46c3-a50e-ede9fc6f250a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:32.625496 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-856e0170-3ebc-46c3-a50e-ede9fc6f250a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:32.643074 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e0170-3ebc-46c3-a50e-ede9fc6f250a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:32.648637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f43708-c488-4383-a7e1-63b0ae33ed04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:32.649261 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f43708-c488-4383-a7e1-63b0ae33ed04 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:32.649627 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f43708-c488-4383-a7e1-63b0ae33ed04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:32.649805 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f43708-c488-4383-a7e1-63b0ae33ed04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:32.658156 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f43708-c488-4383-a7e1-63b0ae33ed04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:32.659765 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f43708-c488-4383-a7e1-63b0ae33ed04 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:32.671100 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1019/2036] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:14:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:14:32.821112 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4390c-6fc3-4e27-b69b-2683839d48f9 None None] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:32.821560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4390c-6fc3-4e27-b69b-2683839d48f9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:32.822051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4390c-6fc3-4e27-b69b-2683839d48f9 None None] PATH_INFO: `/` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:32.823505 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1018/2037] 10.222.0.27 () {54 vars in 921 bytes} [Mon Oct 17 03:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:14:32.829401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5fbff-dec1-4ff3-a362-98b745eeeeff None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:32.829757 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5fbff-dec1-4ff3-a362-98b745eeeeff None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:32.830100 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5fbff-dec1-4ff3-a362-98b745eeeeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:32.851223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a0c5fbff-dec1-4ff3-a362-98b745eeeeff None None] MFA Rules not processed for user `01637fc07ba846aebcb42325bed46216`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:14:32.852427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a0c5fbff-dec1-4ff3-a362-98b745eeeeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:32.861710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1020/2038] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Oct 17 03:14:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:14:32.873168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4390c-6fc3-4e27-b69b-2683839d48f9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:32.884058 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4390c-6fc3-4e27-b69b-2683839d48f9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:32.887754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70846896-5b36-4b48-a243-0acb9db58f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:32.888353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70846896-5b36-4b48-a243-0acb9db58f96 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:32.888633 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70846896-5b36-4b48-a243-0acb9db58f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:32.888907 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70846896-5b36-4b48-a243-0acb9db58f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:32.899112 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70846896-5b36-4b48-a243-0acb9db58f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:32.900541 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70846896-5b36-4b48-a243-0acb9db58f96 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:32.912533 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1019/2039] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:14:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:14:42.845093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=49 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:14:42.906789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=93 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:14:43.283129 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b211d-47c4-4fb8-b325-9acf085386cf None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:43.283916 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b211d-47c4-4fb8-b325-9acf085386cf None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:43.283916 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b211d-47c4-4fb8-b325-9acf085386cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:43.320050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-316b211d-47c4-4fb8-b325-9acf085386cf None None] MFA Rules not processed for user `280d47ed79f14105a83b18b2cac58b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:14:43.324024 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-316b211d-47c4-4fb8-b325-9acf085386cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:43.403901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1021/2040] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Oct 17 03:14:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:14:43.417125 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70846896-5b36-4b48-a243-0acb9db58f96 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:43.418553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-70846896-5b36-4b48-a243-0acb9db58f96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:43.497960 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70846896-5b36-4b48-a243-0acb9db58f96 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:43.502884 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e03ae-9832-4f68-b43c-9b10a161873d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eceda8b3a8ba4233851d0138d8f4d06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:43.503510 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e03ae-9832-4f68-b43c-9b10a161873d service swift] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:43.503792 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e03ae-9832-4f68-b43c-9b10a161873d service swift] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:43.504063 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e03ae-9832-4f68-b43c-9b10a161873d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:43.513065 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e03ae-9832-4f68-b43c-9b10a161873d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:43.514724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e03ae-9832-4f68-b43c-9b10a161873d service swift] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:43.524035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1020/2041] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Oct 17 03:14:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:14:53.393728 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=25 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:14:53.519857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=32 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:14:54.324571 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53485393-981d-4d0b-9add-b94d1958a7b0 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:54.324856 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53485393-981d-4d0b-9add-b94d1958a7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:54.325127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53485393-981d-4d0b-9add-b94d1958a7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:54.344335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-53485393-981d-4d0b-9add-b94d1958a7b0 None None] MFA Rules not processed for user `7cc092c3cc0b4fbc9b420b8c7916132a`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:14:54.345511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-53485393-981d-4d0b-9add-b94d1958a7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:54.402270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1022/2042] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:14:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:14:56.111284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e03ae-9832-4f68-b43c-9b10a161873d service swift] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:14:56.113396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-801e03ae-9832-4f68-b43c-9b10a161873d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:56.186536 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e03ae-9832-4f68-b43c-9b10a161873d service swift] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:14:56.191724 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af366d-c3af-47f0-9676-8b413565be5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cc2655cc32e4d63a7000ef41bcbcfea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:14:56.192351 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af366d-c3af-47f0-9676-8b413565be5b service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:14:56.192749 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af366d-c3af-47f0-9676-8b413565be5b service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:14:56.193040 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af366d-c3af-47f0-9676-8b413565be5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:14:56.195490 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a2af366d-c3af-47f0-9676-8b413565be5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:14:56.208080 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af366d-c3af-47f0-9676-8b413565be5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:14:56.209885 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af366d-c3af-47f0-9676-8b413565be5b service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:14:56.221470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1021/2043] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:14:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:15:04.393978 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=25 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:15:05.903729 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53485393-981d-4d0b-9add-b94d1958a7b0 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:15:05.966083 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53485393-981d-4d0b-9add-b94d1958a7b0 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:15:05.971290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f72c44-2afb-4ae9-9fea-f5c92a78067e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:15:05.971923 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f72c44-2afb-4ae9-9fea-f5c92a78067e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:15:05.972229 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f72c44-2afb-4ae9-9fea-f5c92a78067e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:15:05.972506 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f72c44-2afb-4ae9-9fea-f5c92a78067e admin admin] PATH_INFO: `/v3/users/8229b889af004764b026600e9fb9b6b9` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:15:05.973639 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f72c44-2afb-4ae9-9fea-f5c92a78067e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8229b889af004764b026600e9fb9b6b9)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:15:05.975756 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f72c44-2afb-4ae9-9fea-f5c92a78067e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:15:06.023386 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-97f72c44-2afb-4ae9-9fea-f5c92a78067e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8229b889af004764b026600e9fb9b6b9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:06.030814 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-97f72c44-2afb-4ae9-9fea-f5c92a78067e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8229b889af004764b026600e9fb9b6b9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:06.031590 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-97f72c44-2afb-4ae9-9fea-f5c92a78067e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8229b889af004764b026600e9fb9b6b9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:06.040720 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-97f72c44-2afb-4ae9-9fea-f5c92a78067e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8229b889af004764b026600e9fb9b6b9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:06.054709 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-97f72c44-2afb-4ae9-9fea-f5c92a78067e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8229b889af004764b026600e9fb9b6b9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:06.062234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1023/2044] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:15:05 2022] DELETE /identity/v3/users/8229b889af004764b026600e9fb9b6b9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:15:06.066500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8737b-a1f8-4133-a55c-e8d110b3f1eb None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:15:06.067484 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8737b-a1f8-4133-a55c-e8d110b3f1eb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:15:06.067843 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8737b-a1f8-4133-a55c-e8d110b3f1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:15:06.093141 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-cbf8737b-a1f8-4133-a55c-e8d110b3f1eb None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:15:06.094582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cbf8737b-a1f8-4133-a55c-e8d110b3f1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:15:06.164978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1022/2045] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:15:06.174808 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f72c44-2afb-4ae9-9fea-f5c92a78067e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:15:06.176337 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-97f72c44-2afb-4ae9-9fea-f5c92a78067e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:15:06.239450 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f72c44-2afb-4ae9-9fea-f5c92a78067e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:15:06.245585 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842678f8-c711-4b02-a373-a3667a5ede4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:15:06.246336 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842678f8-c711-4b02-a373-a3667a5ede4b service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:15:06.246730 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842678f8-c711-4b02-a373-a3667a5ede4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:15:06.247072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842678f8-c711-4b02-a373-a3667a5ede4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:15:06.258179 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842678f8-c711-4b02-a373-a3667a5ede4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:15:06.259677 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842678f8-c711-4b02-a373-a3667a5ede4b service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:15:06.269937 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1024/2046] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:15:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:15:06.636441 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8737b-a1f8-4133-a55c-e8d110b3f1eb None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:15:06.637982 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cbf8737b-a1f8-4133-a55c-e8d110b3f1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:15:06.650401 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf8737b-a1f8-4133-a55c-e8d110b3f1eb None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:15:06.655036 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1a9ef-2c3a-42f7-8420-7258842452ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:15:06.655933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1a9ef-2c3a-42f7-8420-7258842452ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:15:06.656213 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1a9ef-2c3a-42f7-8420-7258842452ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:15:06.656490 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1a9ef-2c3a-42f7-8420-7258842452ec admin admin] PATH_INFO: `/v3/projects/862ff19a5e444868be6cc4cfa3a11cfb` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:15:06.657641 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1a9ef-2c3a-42f7-8420-7258842452ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=862ff19a5e444868be6cc4cfa3a11cfb)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:15:06.659596 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1a9ef-2c3a-42f7-8420-7258842452ec admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:15:06.707057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-44e1a9ef-2c3a-42f7-8420-7258842452ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '862ff19a5e444868be6cc4cfa3a11cfb', 'request_id': 'req-44e1a9ef-2c3a-42f7-8420-7258842452ec'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:06.713290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-44e1a9ef-2c3a-42f7-8420-7258842452ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '862ff19a5e444868be6cc4cfa3a11cfb', 'request_id': 'req-44e1a9ef-2c3a-42f7-8420-7258842452ec'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:06.719430 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-44e1a9ef-2c3a-42f7-8420-7258842452ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '862ff19a5e444868be6cc4cfa3a11cfb', 'request_id': 'req-44e1a9ef-2c3a-42f7-8420-7258842452ec'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:06.732523 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-44e1a9ef-2c3a-42f7-8420-7258842452ec admin admin] The token cache is being invalidate because project 862ff19a5e444868be6cc4cfa3a11cfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:15:06.732816 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-44e1a9ef-2c3a-42f7-8420-7258842452ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:06.733441 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-44e1a9ef-2c3a-42f7-8420-7258842452ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:06.734971 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1023/2047] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:15:06 2022] DELETE /identity/v3/projects/862ff19a5e444868be6cc4cfa3a11cfb => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:15:09.048738 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4b262-f593-4180-8a95-41f8f6891b18 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:15:09.049205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4b262-f593-4180-8a95-41f8f6891b18 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:15:09.049837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4b262-f593-4180-8a95-41f8f6891b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:15:09.070436 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5ed4b262-f593-4180-8a95-41f8f6891b18 None None] MFA Rules not processed for user `280d47ed79f14105a83b18b2cac58b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:15:09.071614 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5ed4b262-f593-4180-8a95-41f8f6891b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:15:09.088763 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:15:09.127467 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1025/2048] 10.222.0.27 () {58 vars in 967 bytes} [Mon Oct 17 03:15:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:15:09.150346 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1a9ef-2c3a-42f7-8420-7258842452ec admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:15:09.151739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-44e1a9ef-2c3a-42f7-8420-7258842452ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:15:09.166369 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:15:09.223202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1a9ef-2c3a-42f7-8420-7258842452ec admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:15:09.228276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfdf65-0b44-46de-9431-76259a69d483 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eceda8b3a8ba4233851d0138d8f4d06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:15:09.228897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfdf65-0b44-46de-9431-76259a69d483 service swift] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:15:09.229178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfdf65-0b44-46de-9431-76259a69d483 service swift] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:15:09.229451 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfdf65-0b44-46de-9431-76259a69d483 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:15:09.237354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bfdf65-0b44-46de-9431-76259a69d483 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:15:09.238956 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bfdf65-0b44-46de-9431-76259a69d483 service swift] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:15:09.247045 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1024/2049] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Oct 17 03:15:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:15:19.120839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=15 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:15:19.243849 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=26 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:15:55.924149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4b262-f593-4180-8a95-41f8f6891b18 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:15:55.926257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5ed4b262-f593-4180-8a95-41f8f6891b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:15:56.007928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4b262-f593-4180-8a95-41f8f6891b18 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:15:56.007928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff589f3-bb1b-4b3c-a58d-675f2b71ed3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:15:56.007928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff589f3-bb1b-4b3c-a58d-675f2b71ed3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:15:56.007928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff589f3-bb1b-4b3c-a58d-675f2b71ed3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:15:56.008303 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff589f3-bb1b-4b3c-a58d-675f2b71ed3a admin admin] PATH_INFO: `/v3/users/7cc092c3cc0b4fbc9b420b8c7916132a` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:15:56.010173 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff589f3-bb1b-4b3c-a58d-675f2b71ed3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cc092c3cc0b4fbc9b420b8c7916132a)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:15:56.012348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff589f3-bb1b-4b3c-a58d-675f2b71ed3a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:15:56.059019 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7ff589f3-bb1b-4b3c-a58d-675f2b71ed3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cc092c3cc0b4fbc9b420b8c7916132a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:56.066319 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7ff589f3-bb1b-4b3c-a58d-675f2b71ed3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cc092c3cc0b4fbc9b420b8c7916132a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:56.067031 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7ff589f3-bb1b-4b3c-a58d-675f2b71ed3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cc092c3cc0b4fbc9b420b8c7916132a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:56.076349 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7ff589f3-bb1b-4b3c-a58d-675f2b71ed3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cc092c3cc0b4fbc9b420b8c7916132a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:56.093190 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7ff589f3-bb1b-4b3c-a58d-675f2b71ed3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cc092c3cc0b4fbc9b420b8c7916132a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:56.101701 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1026/2050] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:15:55 2022] DELETE /identity/v3/users/7cc092c3cc0b4fbc9b420b8c7916132a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:15:56.287500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfdf65-0b44-46de-9431-76259a69d483 service swift] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:15:56.299717 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfdf65-0b44-46de-9431-76259a69d483 service swift] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:15:56.362196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42ad3a-4beb-4912-8ea0-408fcd6c2575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:15:56.362841 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42ad3a-4beb-4912-8ea0-408fcd6c2575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:15:56.363199 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42ad3a-4beb-4912-8ea0-408fcd6c2575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:15:56.363670 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42ad3a-4beb-4912-8ea0-408fcd6c2575 admin admin] PATH_INFO: `/v3/projects/0d285ecc0c5343b1b4001594013cd447` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:15:56.364821 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42ad3a-4beb-4912-8ea0-408fcd6c2575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d285ecc0c5343b1b4001594013cd447)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:15:56.366734 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42ad3a-4beb-4912-8ea0-408fcd6c2575 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:15:56.415935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ee42ad3a-4beb-4912-8ea0-408fcd6c2575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d285ecc0c5343b1b4001594013cd447', 'request_id': 'req-ee42ad3a-4beb-4912-8ea0-408fcd6c2575'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:56.421557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ee42ad3a-4beb-4912-8ea0-408fcd6c2575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d285ecc0c5343b1b4001594013cd447', 'request_id': 'req-ee42ad3a-4beb-4912-8ea0-408fcd6c2575'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:56.428419 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ee42ad3a-4beb-4912-8ea0-408fcd6c2575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d285ecc0c5343b1b4001594013cd447', 'request_id': 'req-ee42ad3a-4beb-4912-8ea0-408fcd6c2575'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:56.450263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ee42ad3a-4beb-4912-8ea0-408fcd6c2575 admin admin] The token cache is being invalidate because project 0d285ecc0c5343b1b4001594013cd447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:15:56.450263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ee42ad3a-4beb-4912-8ea0-408fcd6c2575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:56.450263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ee42ad3a-4beb-4912-8ea0-408fcd6c2575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:15:56.450263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1025/2051] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:15:56 2022] DELETE /identity/v3/projects/0d285ecc0c5343b1b4001594013cd447 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:15:56.456676 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cfc89-a7c3-4a28-be29-dd8b785d6a9a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:15:56.456997 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cfc89-a7c3-4a28-be29-dd8b785d6a9a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:15:56.457270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cfc89-a7c3-4a28-be29-dd8b785d6a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:15:56.478574 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-959cfc89-a7c3-4a28-be29-dd8b785d6a9a None None] MFA Rules not processed for user `1c786544e8fb4e8e8c3a4990a0f36725`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:15:56.479846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-959cfc89-a7c3-4a28-be29-dd8b785d6a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:15:56.543817 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1027/2052] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:15:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:16:06.438776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:16:06.536178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:16:09.381519 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a9ac9-2155-4078-aec2-0d81ef2c7c11 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:16:09.381519 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a9ac9-2155-4078-aec2-0d81ef2c7c11 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:16:09.381935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a9ac9-2155-4078-aec2-0d81ef2c7c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:16:09.401269 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3d0a9ac9-2155-4078-aec2-0d81ef2c7c11 None None] MFA Rules not processed for user `37e6d36d529146299970874660e92e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:16:09.402701 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3d0a9ac9-2155-4078-aec2-0d81ef2c7c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:16:09.456908 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1026/2053] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:16:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:16:19.450122 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=25 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:16:27.960491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3695406-eb8e-471c-9394-f73fcad90a06 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:16:27.961026 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3695406-eb8e-471c-9394-f73fcad90a06 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:16:27.961026 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3695406-eb8e-471c-9394-f73fcad90a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:16:27.980527 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a3695406-eb8e-471c-9394-f73fcad90a06 None None] MFA Rules not processed for user `9c1787a9ef7c44e28076aaa80c0152fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:16:27.981711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a3695406-eb8e-471c-9394-f73fcad90a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:16:28.035289 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1028/2054] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:16:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:16:35.572549 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fb3b4-9e31-4e84-90a1-4f3950c6e9ff None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:16:35.573142 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fb3b4-9e31-4e84-90a1-4f3950c6e9ff None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:16:35.573142 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fb3b4-9e31-4e84-90a1-4f3950c6e9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:16:35.592342 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0d4fb3b4-9e31-4e84-90a1-4f3950c6e9ff None None] MFA Rules not processed for user `ce25db99a7b14d0bbaca41fff80e3da9`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:16:35.593556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0d4fb3b4-9e31-4e84-90a1-4f3950c6e9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:16:35.649569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1027/2055] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:16:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:16:37.750633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3695406-eb8e-471c-9394-f73fcad90a06 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:16:37.752349 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a3695406-eb8e-471c-9394-f73fcad90a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:16:37.817668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3695406-eb8e-471c-9394-f73fcad90a06 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:16:37.821881 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d10f8-845f-4f2c-b283-00939a036455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:16:37.822563 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d10f8-845f-4f2c-b283-00939a036455 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:16:37.822843 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d10f8-845f-4f2c-b283-00939a036455 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:16:37.823167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d10f8-845f-4f2c-b283-00939a036455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:16:37.825045 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-110d10f8-845f-4f2c-b283-00939a036455 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:16:37.910419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d10f8-845f-4f2c-b283-00939a036455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:16:37.912472 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d10f8-845f-4f2c-b283-00939a036455 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:16:37.926893 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1029/2056] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:16:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:16:39.058353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fb3b4-9e31-4e84-90a1-4f3950c6e9ff None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:16:39.060002 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0d4fb3b4-9e31-4e84-90a1-4f3950c6e9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:16:39.072119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fb3b4-9e31-4e84-90a1-4f3950c6e9ff None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:16:39.075679 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00937a03-0c66-496b-adfb-f72741cb2bad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:16:39.076284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00937a03-0c66-496b-adfb-f72741cb2bad service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:16:39.076570 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00937a03-0c66-496b-adfb-f72741cb2bad service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:16:39.076876 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00937a03-0c66-496b-adfb-f72741cb2bad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:16:39.078600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-00937a03-0c66-496b-adfb-f72741cb2bad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:16:39.092373 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00937a03-0c66-496b-adfb-f72741cb2bad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:16:39.093981 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00937a03-0c66-496b-adfb-f72741cb2bad service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:16:39.105903 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1028/2057] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:16:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:16:47.921235 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=79 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:16:49.100122 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=31 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:16:56.763094 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d10f8-845f-4f2c-b283-00939a036455 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:16:56.764512 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-110d10f8-845f-4f2c-b283-00939a036455 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:16:56.778230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d10f8-845f-4f2c-b283-00939a036455 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:16:56.783922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef5fbc-6a88-4025-947c-094c4b06dbac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:16:56.784509 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef5fbc-6a88-4025-947c-094c4b06dbac service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:16:56.784786 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef5fbc-6a88-4025-947c-094c4b06dbac service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:16:56.785056 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef5fbc-6a88-4025-947c-094c4b06dbac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:16:56.786717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-60ef5fbc-6a88-4025-947c-094c4b06dbac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:16:56.797070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef5fbc-6a88-4025-947c-094c4b06dbac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:16:56.798654 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef5fbc-6a88-4025-947c-094c4b06dbac service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:16:56.808706 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1030/2058] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:16:56 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:16:59.592426 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00937a03-0c66-496b-adfb-f72741cb2bad service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:16:59.601975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00937a03-0c66-496b-adfb-f72741cb2bad service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:16:59.606447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf16dc-2b9e-478c-8184-f9c959feb221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:16:59.607090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf16dc-2b9e-478c-8184-f9c959feb221 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:16:59.607371 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf16dc-2b9e-478c-8184-f9c959feb221 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:16:59.607649 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf16dc-2b9e-478c-8184-f9c959feb221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:16:59.609452 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4ddf16dc-2b9e-478c-8184-f9c959feb221 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:16:59.622228 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf16dc-2b9e-478c-8184-f9c959feb221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:16:59.623865 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf16dc-2b9e-478c-8184-f9c959feb221 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:16:59.640967 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1029/2059] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:16:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:17:00.575189 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712961ef-68e1-4434-8308-b2d80de91e1f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:00.575683 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712961ef-68e1-4434-8308-b2d80de91e1f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:00.575785 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712961ef-68e1-4434-8308-b2d80de91e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:00.594581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-712961ef-68e1-4434-8308-b2d80de91e1f None None] MFA Rules not processed for user `1c786544e8fb4e8e8c3a4990a0f36725`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:00.595790 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-712961ef-68e1-4434-8308-b2d80de91e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:00.602856 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1031/2060] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:17:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:17:00.901920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf16dc-2b9e-478c-8184-f9c959feb221 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:00.910561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf16dc-2b9e-478c-8184-f9c959feb221 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:00.914180 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796afa9b-6c6b-4a51-a55e-89731d1cca77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:00.915351 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796afa9b-6c6b-4a51-a55e-89731d1cca77 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:00.915616 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796afa9b-6c6b-4a51-a55e-89731d1cca77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:00.915886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796afa9b-6c6b-4a51-a55e-89731d1cca77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:00.917585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-796afa9b-6c6b-4a51-a55e-89731d1cca77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:00.926854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796afa9b-6c6b-4a51-a55e-89731d1cca77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:00.928300 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796afa9b-6c6b-4a51-a55e-89731d1cca77 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:00.936786 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1030/2061] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:17:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:17:10.589082 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=9 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:10.933357 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=12 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:11.819704 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712961ef-68e1-4434-8308-b2d80de91e1f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:11.829296 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712961ef-68e1-4434-8308-b2d80de91e1f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:11.833017 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b2ee7-f261-42b3-97dc-4d2f71198525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:11.833621 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b2ee7-f261-42b3-97dc-4d2f71198525 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:11.833902 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b2ee7-f261-42b3-97dc-4d2f71198525 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:11.834180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b2ee7-f261-42b3-97dc-4d2f71198525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:11.835962 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-028b2ee7-f261-42b3-97dc-4d2f71198525 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:11.847488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b2ee7-f261-42b3-97dc-4d2f71198525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:11.848954 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b2ee7-f261-42b3-97dc-4d2f71198525 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:11.857739 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1032/2062] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:17:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:17:12.633408 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796afa9b-6c6b-4a51-a55e-89731d1cca77 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:12.636381 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-796afa9b-6c6b-4a51-a55e-89731d1cca77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:12.699086 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796afa9b-6c6b-4a51-a55e-89731d1cca77 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:12.703096 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64554dd-19a7-4354-9982-f36d1417e656 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:12.703935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64554dd-19a7-4354-9982-f36d1417e656 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:12.704069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64554dd-19a7-4354-9982-f36d1417e656 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:12.704241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64554dd-19a7-4354-9982-f36d1417e656 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:12.705997 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a64554dd-19a7-4354-9982-f36d1417e656 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:12.717974 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64554dd-19a7-4354-9982-f36d1417e656 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:12.719451 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64554dd-19a7-4354-9982-f36d1417e656 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:12.729326 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1031/2063] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:17:12 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:17:14.031864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a8725-94a6-47e4-b4ce-47e9f186beb2 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:14.032358 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a8725-94a6-47e4-b4ce-47e9f186beb2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:14.032455 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a8725-94a6-47e4-b4ce-47e9f186beb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:14.053147 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-cd6a8725-94a6-47e4-b4ce-47e9f186beb2 None None] MFA Rules not processed for user `280d47ed79f14105a83b18b2cac58b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:14.054349 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cd6a8725-94a6-47e4-b4ce-47e9f186beb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:14.112923 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1033/2064] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Oct 17 03:17:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:17:14.162017 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64554dd-19a7-4354-9982-f36d1417e656 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:14.163566 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a64554dd-19a7-4354-9982-f36d1417e656 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:14.243860 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64554dd-19a7-4354-9982-f36d1417e656 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:14.249392 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bff758-f157-4c2f-8375-d67a445aa964 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eceda8b3a8ba4233851d0138d8f4d06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:14.250126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bff758-f157-4c2f-8375-d67a445aa964 service swift] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:14.250606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bff758-f157-4c2f-8375-d67a445aa964 service swift] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:14.251197 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bff758-f157-4c2f-8375-d67a445aa964 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:14.265167 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bff758-f157-4c2f-8375-d67a445aa964 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:14.267086 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bff758-f157-4c2f-8375-d67a445aa964 service swift] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:14.282479 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1032/2065] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Oct 17 03:17:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:17:24.105706 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=31 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:24.272664 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=58 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:27.144020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a8725-94a6-47e4-b4ce-47e9f186beb2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:27.187791 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a8725-94a6-47e4-b4ce-47e9f186beb2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:27.195658 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702ac56-483f-4e85-acd2-ae54e5924dd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:27.197070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702ac56-483f-4e85-acd2-ae54e5924dd4 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:27.197603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702ac56-483f-4e85-acd2-ae54e5924dd4 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:27.199119 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702ac56-483f-4e85-acd2-ae54e5924dd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:27.201956 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9702ac56-483f-4e85-acd2-ae54e5924dd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:27.216190 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702ac56-483f-4e85-acd2-ae54e5924dd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:27.217979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702ac56-483f-4e85-acd2-ae54e5924dd4 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:27.230751 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1034/2066] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:17:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:17:30.368934 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bff758-f157-4c2f-8375-d67a445aa964 service swift] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:30.370325 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e6bff758-f157-4c2f-8375-d67a445aa964 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:30.438780 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bff758-f157-4c2f-8375-d67a445aa964 service swift] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:30.443686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0726b8-9ecd-413c-a802-fbc01d619dd0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cc2655cc32e4d63a7000ef41bcbcfea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:30.444323 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0726b8-9ecd-413c-a802-fbc01d619dd0 service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:30.444603 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0726b8-9ecd-413c-a802-fbc01d619dd0 service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:30.444873 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0726b8-9ecd-413c-a802-fbc01d619dd0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:30.447529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dd0726b8-9ecd-413c-a802-fbc01d619dd0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:30.458892 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0726b8-9ecd-413c-a802-fbc01d619dd0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:30.460470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0726b8-9ecd-413c-a802-fbc01d619dd0 service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:30.470436 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1033/2067] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:17:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:17:32.845558 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702ac56-483f-4e85-acd2-ae54e5924dd4 service nova] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:32.856664 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702ac56-483f-4e85-acd2-ae54e5924dd4 service nova] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:32.862097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852019c-007b-4ae1-b88d-43a6fb131488 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:32.862721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852019c-007b-4ae1-b88d-43a6fb131488 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:32.863044 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852019c-007b-4ae1-b88d-43a6fb131488 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:32.863320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852019c-007b-4ae1-b88d-43a6fb131488 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:32.865027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0852019c-007b-4ae1-b88d-43a6fb131488 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:32.875603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852019c-007b-4ae1-b88d-43a6fb131488 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:32.877167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852019c-007b-4ae1-b88d-43a6fb131488 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:32.886489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1035/2068] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:17:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:17:34.598638 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bccf5-0dcb-457b-93c4-359667e1fc41 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:34.599147 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bccf5-0dcb-457b-93c4-359667e1fc41 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:34.599328 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bccf5-0dcb-457b-93c4-359667e1fc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:34.624750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7d5bccf5-0dcb-457b-93c4-359667e1fc41 None None] MFA Rules not processed for user `280d47ed79f14105a83b18b2cac58b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:34.626057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7d5bccf5-0dcb-457b-93c4-359667e1fc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:34.633431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1034/2069] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Oct 17 03:17:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:17:34.650534 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852019c-007b-4ae1-b88d-43a6fb131488 service nova] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:34.664126 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852019c-007b-4ae1-b88d-43a6fb131488 service nova] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:34.670262 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483066a9-c8af-462d-85c4-a75c30d64100 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eceda8b3a8ba4233851d0138d8f4d06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:34.670889 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483066a9-c8af-462d-85c4-a75c30d64100 service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:34.671179 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483066a9-c8af-462d-85c4-a75c30d64100 service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:34.671455 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483066a9-c8af-462d-85c4-a75c30d64100 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:34.683997 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483066a9-c8af-462d-85c4-a75c30d64100 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:34.685849 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483066a9-c8af-462d-85c4-a75c30d64100 service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:34.694726 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1036/2070] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Oct 17 03:17:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:17:34.763404 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bccf5-0dcb-457b-93c4-359667e1fc41 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:34.773241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bccf5-0dcb-457b-93c4-359667e1fc41 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:34.777823 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd86c9-fb1f-400f-9761-e0040da8a609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:34.778444 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd86c9-fb1f-400f-9761-e0040da8a609 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:34.778745 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd86c9-fb1f-400f-9761-e0040da8a609 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:34.779056 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd86c9-fb1f-400f-9761-e0040da8a609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:34.780920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-adbd86c9-fb1f-400f-9761-e0040da8a609 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:34.878933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd86c9-fb1f-400f-9761-e0040da8a609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:34.880830 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd86c9-fb1f-400f-9761-e0040da8a609 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:34.897076 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1035/2071] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:17:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:17:38.917206 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483066a9-c8af-462d-85c4-a75c30d64100 service swift] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:38.922650 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-483066a9-c8af-462d-85c4-a75c30d64100 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:38.952098 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483066a9-c8af-462d-85c4-a75c30d64100 service swift] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:38.965165 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67a826-b2bc-4c6c-9366-4b7787db5ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:38.966081 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67a826-b2bc-4c6c-9366-4b7787db5ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:38.966550 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67a826-b2bc-4c6c-9366-4b7787db5ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:38.967031 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67a826-b2bc-4c6c-9366-4b7787db5ab9 admin admin] PATH_INFO: `/v3/users/9c1787a9ef7c44e28076aaa80c0152fd` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:38.974885 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67a826-b2bc-4c6c-9366-4b7787db5ab9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c1787a9ef7c44e28076aaa80c0152fd)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:38.977088 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67a826-b2bc-4c6c-9366-4b7787db5ab9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:39.045587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bf67a826-b2bc-4c6c-9366-4b7787db5ab9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c1787a9ef7c44e28076aaa80c0152fd'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:39.053517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bf67a826-b2bc-4c6c-9366-4b7787db5ab9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c1787a9ef7c44e28076aaa80c0152fd'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:39.054507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bf67a826-b2bc-4c6c-9366-4b7787db5ab9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c1787a9ef7c44e28076aaa80c0152fd'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:39.063858 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bf67a826-b2bc-4c6c-9366-4b7787db5ab9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c1787a9ef7c44e28076aaa80c0152fd'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:39.076965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-bf67a826-b2bc-4c6c-9366-4b7787db5ab9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c1787a9ef7c44e28076aaa80c0152fd'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:39.083313 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1037/2072] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:17:38 2022] DELETE /identity/v3/users/9c1787a9ef7c44e28076aaa80c0152fd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:39.300685 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd86c9-fb1f-400f-9761-e0040da8a609 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:39.336781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd86c9-fb1f-400f-9761-e0040da8a609 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:39.422144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb19ce-04d8-48d8-9ac7-6ee82c09bb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:39.422819 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb19ce-04d8-48d8-9ac7-6ee82c09bb43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:39.423202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb19ce-04d8-48d8-9ac7-6ee82c09bb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:39.423650 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb19ce-04d8-48d8-9ac7-6ee82c09bb43 admin admin] PATH_INFO: `/v3/projects/ee21a91b84794e80b208fe72b7f1e010` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:39.425190 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb19ce-04d8-48d8-9ac7-6ee82c09bb43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee21a91b84794e80b208fe72b7f1e010)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:39.427374 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb19ce-04d8-48d8-9ac7-6ee82c09bb43 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:39.452194 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:39.486001 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7cfb19ce-04d8-48d8-9ac7-6ee82c09bb43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee21a91b84794e80b208fe72b7f1e010', 'request_id': 'req-7cfb19ce-04d8-48d8-9ac7-6ee82c09bb43'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:39.492106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7cfb19ce-04d8-48d8-9ac7-6ee82c09bb43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee21a91b84794e80b208fe72b7f1e010', 'request_id': 'req-7cfb19ce-04d8-48d8-9ac7-6ee82c09bb43'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:39.500428 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7cfb19ce-04d8-48d8-9ac7-6ee82c09bb43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee21a91b84794e80b208fe72b7f1e010', 'request_id': 'req-7cfb19ce-04d8-48d8-9ac7-6ee82c09bb43'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:39.515563 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7cfb19ce-04d8-48d8-9ac7-6ee82c09bb43 admin admin] The token cache is being invalidate because project ee21a91b84794e80b208fe72b7f1e010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:17:39.516050 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7cfb19ce-04d8-48d8-9ac7-6ee82c09bb43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:39.516778 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7cfb19ce-04d8-48d8-9ac7-6ee82c09bb43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:39.518695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1036/2073] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:17:39 2022] DELETE /identity/v3/projects/ee21a91b84794e80b208fe72b7f1e010 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:39.523695 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06dcd7-123f-46ca-a699-e500557291b8 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:39.523987 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06dcd7-123f-46ca-a699-e500557291b8 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:39.524266 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06dcd7-123f-46ca-a699-e500557291b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:39.544350 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1b06dcd7-123f-46ca-a699-e500557291b8 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:39.545549 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1b06dcd7-123f-46ca-a699-e500557291b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:39.608353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1038/2074] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:17:39.616420 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0366c-acbd-46aa-8677-a6ffc47ff2f5 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:39.616707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0366c-acbd-46aa-8677-a6ffc47ff2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:39.616983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0366c-acbd-46aa-8677-a6ffc47ff2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:39.638337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ebc0366c-acbd-46aa-8677-a6ffc47ff2f5 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:39.639679 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ebc0366c-acbd-46aa-8677-a6ffc47ff2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:39.648138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1037/2075] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:17:39.652794 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06dcd7-123f-46ca-a699-e500557291b8 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:39.663231 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06dcd7-123f-46ca-a699-e500557291b8 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:39.668243 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80dffc-f55f-4f0d-ae34-c6d5c556bb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:39.668862 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80dffc-f55f-4f0d-ae34-c6d5c556bb24 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:39.669205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80dffc-f55f-4f0d-ae34-c6d5c556bb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:39.669484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80dffc-f55f-4f0d-ae34-c6d5c556bb24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:39.670143 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80dffc-f55f-4f0d-ae34-c6d5c556bb24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:17:39.670453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80dffc-f55f-4f0d-ae34-c6d5c556bb24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:39.672114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80dffc-f55f-4f0d-ae34-c6d5c556bb24 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:39.686889 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1039/2076] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:17:39.690929 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0366c-acbd-46aa-8677-a6ffc47ff2f5 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:39.701438 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0366c-acbd-46aa-8677-a6ffc47ff2f5 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:39.706142 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52522c2a-41f4-4919-84a0-b040956a0073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:39.706752 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52522c2a-41f4-4919-84a0-b040956a0073 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:39.707074 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52522c2a-41f4-4919-84a0-b040956a0073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:39.707353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52522c2a-41f4-4919-84a0-b040956a0073 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:39.708005 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52522c2a-41f4-4919-84a0-b040956a0073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:39.709922 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52522c2a-41f4-4919-84a0-b040956a0073 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:39.744694 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1038/2077] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:17:39 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:17:39.745683 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80dffc-f55f-4f0d-ae34-c6d5c556bb24 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:39.759185 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80dffc-f55f-4f0d-ae34-c6d5c556bb24 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:39.776923 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:39.824096 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399831dd-3a14-4366-b6d0-57e0b860c1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:39.824876 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399831dd-3a14-4366-b6d0-57e0b860c1ed admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:39.825167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399831dd-3a14-4366-b6d0-57e0b860c1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:39.825434 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399831dd-3a14-4366-b6d0-57e0b860c1ed admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:39.826165 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399831dd-3a14-4366-b6d0-57e0b860c1ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:39.828260 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399831dd-3a14-4366-b6d0-57e0b860c1ed admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:39.855551 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1040/2078] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:17:39 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:17:39.856244 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52522c2a-41f4-4919-84a0-b040956a0073 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:39.867577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52522c2a-41f4-4919-84a0-b040956a0073 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:39.874097 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a2e5c-89be-4780-bde5-9d9ddd9db27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:39.874976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a2e5c-89be-4780-bde5-9d9ddd9db27f admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:39.875266 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a2e5c-89be-4780-bde5-9d9ddd9db27f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:39.875531 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a2e5c-89be-4780-bde5-9d9ddd9db27f admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:39.876263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a2e5c-89be-4780-bde5-9d9ddd9db27f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:39.878100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a2e5c-89be-4780-bde5-9d9ddd9db27f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:39.891426 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1039/2079] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:17:39.895457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399831dd-3a14-4366-b6d0-57e0b860c1ed admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:39.906288 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399831dd-3a14-4366-b6d0-57e0b860c1ed admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:39.911137 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f6367-3cc7-4f73-ad11-d9f0e2947100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:39.911737 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f6367-3cc7-4f73-ad11-d9f0e2947100 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:39.912017 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f6367-3cc7-4f73-ad11-d9f0e2947100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:39.912287 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f6367-3cc7-4f73-ad11-d9f0e2947100 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:39.912928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f6367-3cc7-4f73-ad11-d9f0e2947100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:39.914517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f6367-3cc7-4f73-ad11-d9f0e2947100 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:39.927280 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1041/2080] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:17:39.932183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a2e5c-89be-4780-bde5-9d9ddd9db27f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:39.942290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a2e5c-89be-4780-bde5-9d9ddd9db27f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:39.947098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25602a37-a0e9-4ce3-ae15-9739bb592835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:39.947775 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25602a37-a0e9-4ce3-ae15-9739bb592835 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:39.948058 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25602a37-a0e9-4ce3-ae15-9739bb592835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:39.948337 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25602a37-a0e9-4ce3-ae15-9739bb592835 admin admin] PATH_INFO: `/v3/projects/bc61eb62575941baa41b079f1a697971/users/f87a1fed2bb34ee4b30d221043d63a7c/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:39.965679 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25602a37-a0e9-4ce3-ae15-9739bb592835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc61eb62575941baa41b079f1a697971, user_id=f87a1fed2bb34ee4b30d221043d63a7c, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:39.968777 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25602a37-a0e9-4ce3-ae15-9739bb592835 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:39.989756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1040/2081] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:17:39 2022] PUT /identity/v3/projects/bc61eb62575941baa41b079f1a697971/users/f87a1fed2bb34ee4b30d221043d63a7c/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:40.002352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5db10-6d49-4ade-a9a9-7a12cf1b011e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:40.002703 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5db10-6d49-4ade-a9a9-7a12cf1b011e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:40.003050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5db10-6d49-4ade-a9a9-7a12cf1b011e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:40.038639 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2ce5db10-6d49-4ade-a9a9-7a12cf1b011e None None] MFA Rules not processed for user `f87a1fed2bb34ee4b30d221043d63a7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:40.039894 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2ce5db10-6d49-4ade-a9a9-7a12cf1b011e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:40.137058 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1042/2082] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:17:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:17:40.137946 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25602a37-a0e9-4ce3-ae15-9739bb592835 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:40.202493 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25602a37-a0e9-4ce3-ae15-9739bb592835 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:40.207375 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a54814-cb39-4e91-b442-28a1bb6cebcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:40.208110 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a54814-cb39-4e91-b442-28a1bb6cebcf admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:40.211152 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a54814-cb39-4e91-b442-28a1bb6cebcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:40.211511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a54814-cb39-4e91-b442-28a1bb6cebcf admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:40.215806 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a54814-cb39-4e91-b442-28a1bb6cebcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:40.215806 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a54814-cb39-4e91-b442-28a1bb6cebcf admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:40.261753 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1041/2083] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:17:40 2022] POST /identity/v3/projects => generated 363 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:17:40.268137 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5db10-6d49-4ade-a9a9-7a12cf1b011e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:40.283480 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5db10-6d49-4ade-a9a9-7a12cf1b011e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:40.375908 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db441c-1711-4bd7-a7c1-252eefa0b3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:40.376810 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db441c-1711-4bd7-a7c1-252eefa0b3ab admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:40.377264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db441c-1711-4bd7-a7c1-252eefa0b3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:40.377641 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db441c-1711-4bd7-a7c1-252eefa0b3ab admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:40.378441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db441c-1711-4bd7-a7c1-252eefa0b3ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:40.380518 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db441c-1711-4bd7-a7c1-252eefa0b3ab admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:40.424757 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1043/2084] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:17:40 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:17:40.428832 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a54814-cb39-4e91-b442-28a1bb6cebcf admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:40.440321 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a54814-cb39-4e91-b442-28a1bb6cebcf admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:40.445960 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65fe38-8534-492a-8cf3-662bc60fd3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:40.447251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65fe38-8534-492a-8cf3-662bc60fd3ef admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:40.447251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65fe38-8534-492a-8cf3-662bc60fd3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:40.447540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65fe38-8534-492a-8cf3-662bc60fd3ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:40.448130 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65fe38-8534-492a-8cf3-662bc60fd3ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:40.449715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65fe38-8534-492a-8cf3-662bc60fd3ef admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:40.462575 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1042/2085] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:17:40.466719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db441c-1711-4bd7-a7c1-252eefa0b3ab admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:40.476867 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db441c-1711-4bd7-a7c1-252eefa0b3ab admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:40.482788 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e681582-f140-4600-bccd-ab8099bfc28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:40.483499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e681582-f140-4600-bccd-ab8099bfc28b admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:40.483789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e681582-f140-4600-bccd-ab8099bfc28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:40.484025 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e681582-f140-4600-bccd-ab8099bfc28b admin admin] PATH_INFO: `/v3/projects/16d88cf51228474c84ad9d495494115c/users/66d56f4d800445d193c7c0dd62fe87ac/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:40.508745 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e681582-f140-4600-bccd-ab8099bfc28b admin admin] RBAC: Authorizing `identity:create_grant(project_id=16d88cf51228474c84ad9d495494115c, user_id=66d56f4d800445d193c7c0dd62fe87ac, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:40.512710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e681582-f140-4600-bccd-ab8099bfc28b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:40.515154 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:40.533887 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1044/2086] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:17:40 2022] PUT /identity/v3/projects/16d88cf51228474c84ad9d495494115c/users/66d56f4d800445d193c7c0dd62fe87ac/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:40.536219 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65fe38-8534-492a-8cf3-662bc60fd3ef admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:40.546756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65fe38-8534-492a-8cf3-662bc60fd3ef admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:40.559140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:40.612879 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92ffaf-8308-4d7b-8b76-ca308580aa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:40.613693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92ffaf-8308-4d7b-8b76-ca308580aa6e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:40.614097 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92ffaf-8308-4d7b-8b76-ca308580aa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:40.614442 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92ffaf-8308-4d7b-8b76-ca308580aa6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:40.615288 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92ffaf-8308-4d7b-8b76-ca308580aa6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:40.617197 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92ffaf-8308-4d7b-8b76-ca308580aa6e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:40.628100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1043/2087] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:17:40.632290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e681582-f140-4600-bccd-ab8099bfc28b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:40.643727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e681582-f140-4600-bccd-ab8099bfc28b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:40.649390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b1bd6-7d51-40b7-b8d0-9970b5e3db81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:40.650490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b1bd6-7d51-40b7-b8d0-9970b5e3db81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:40.650772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b1bd6-7d51-40b7-b8d0-9970b5e3db81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:40.651084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b1bd6-7d51-40b7-b8d0-9970b5e3db81 admin admin] PATH_INFO: `/v3/projects/16d88cf51228474c84ad9d495494115c/users/66d56f4d800445d193c7c0dd62fe87ac/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:40.653194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b1bd6-7d51-40b7-b8d0-9970b5e3db81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16d88cf51228474c84ad9d495494115c, user_id=66d56f4d800445d193c7c0dd62fe87ac, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:40.656346 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b1bd6-7d51-40b7-b8d0-9970b5e3db81 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:40.669466 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1045/2088] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:17:40 2022] PUT /identity/v3/projects/16d88cf51228474c84ad9d495494115c/users/66d56f4d800445d193c7c0dd62fe87ac/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:40.679519 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b605e7-e56a-475d-9c07-412c0f533df0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:40.679943 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b605e7-e56a-475d-9c07-412c0f533df0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:40.680233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b605e7-e56a-475d-9c07-412c0f533df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:40.715996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-02b605e7-e56a-475d-9c07-412c0f533df0 None None] MFA Rules not processed for user `66d56f4d800445d193c7c0dd62fe87ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:40.717354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-02b605e7-e56a-475d-9c07-412c0f533df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:40.793515 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1044/2089] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:17:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:17:40.805204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b1bd6-7d51-40b7-b8d0-9970b5e3db81 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:40.807917 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-755b1bd6-7d51-40b7-b8d0-9970b5e3db81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:40.877605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b1bd6-7d51-40b7-b8d0-9970b5e3db81 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:40.881655 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f78336-3609-451e-85b7-731a00ae9c2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:40.882276 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f78336-3609-451e-85b7-731a00ae9c2f service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:40.882581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f78336-3609-451e-85b7-731a00ae9c2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:40.882860 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f78336-3609-451e-85b7-731a00ae9c2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:40.892858 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f78336-3609-451e-85b7-731a00ae9c2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:40.894307 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f78336-3609-451e-85b7-731a00ae9c2f service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:40.905110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1046/2090] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:17:40 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:17:40.988575 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b605e7-e56a-475d-9c07-412c0f533df0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:41.010765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b605e7-e56a-475d-9c07-412c0f533df0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:41.014514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515526c-523f-4df0-86e9-b4bdd3b755db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:41.015182 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515526c-523f-4df0-86e9-b4bdd3b755db service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:41.015459 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515526c-523f-4df0-86e9-b4bdd3b755db service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:41.015735 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515526c-523f-4df0-86e9-b4bdd3b755db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:41.087613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515526c-523f-4df0-86e9-b4bdd3b755db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:41.089604 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9515526c-523f-4df0-86e9-b4bdd3b755db service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:41.100220 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1045/2091] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:17:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:17:50.899869 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:INSERT=2,keystone:SELECT=33 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:51.095256 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=83 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:51.507278 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f78336-3609-451e-85b7-731a00ae9c2f service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:51.572122 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f78336-3609-451e-85b7-731a00ae9c2f service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:51.576772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d2d75-289b-4375-a05b-1d209d935c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:51.577414 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d2d75-289b-4375-a05b-1d209d935c04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:51.577688 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d2d75-289b-4375-a05b-1d209d935c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:51.578054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d2d75-289b-4375-a05b-1d209d935c04 admin admin] PATH_INFO: `/v3/users/f87a1fed2bb34ee4b30d221043d63a7c` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:51.579237 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d2d75-289b-4375-a05b-1d209d935c04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f87a1fed2bb34ee4b30d221043d63a7c)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:51.581174 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d2d75-289b-4375-a05b-1d209d935c04 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:51.632566 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-627d2d75-289b-4375-a05b-1d209d935c04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f87a1fed2bb34ee4b30d221043d63a7c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:51.640754 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-627d2d75-289b-4375-a05b-1d209d935c04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f87a1fed2bb34ee4b30d221043d63a7c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:51.641540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-627d2d75-289b-4375-a05b-1d209d935c04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f87a1fed2bb34ee4b30d221043d63a7c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:51.649516 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-627d2d75-289b-4375-a05b-1d209d935c04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f87a1fed2bb34ee4b30d221043d63a7c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:51.662405 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-627d2d75-289b-4375-a05b-1d209d935c04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f87a1fed2bb34ee4b30d221043d63a7c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:51.668517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1047/2092] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:17:51 2022] DELETE /identity/v3/users/f87a1fed2bb34ee4b30d221043d63a7c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:51.673191 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27633e0a-6be1-4eea-9d07-be03172f649d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:51.673483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27633e0a-6be1-4eea-9d07-be03172f649d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:51.673817 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27633e0a-6be1-4eea-9d07-be03172f649d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:51.695193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-27633e0a-6be1-4eea-9d07-be03172f649d None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:51.697058 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-27633e0a-6be1-4eea-9d07-be03172f649d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:51.763372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1046/2093] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:17:51.773144 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d2d75-289b-4375-a05b-1d209d935c04 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:51.774957 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-627d2d75-289b-4375-a05b-1d209d935c04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:51.834408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d2d75-289b-4375-a05b-1d209d935c04 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:51.838147 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1bf9f-25cb-4bde-965d-005f7291b273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:51.838786 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1bf9f-25cb-4bde-965d-005f7291b273 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:51.839108 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1bf9f-25cb-4bde-965d-005f7291b273 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:51.839377 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1bf9f-25cb-4bde-965d-005f7291b273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:51.848781 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1bf9f-25cb-4bde-965d-005f7291b273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:51.850246 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1bf9f-25cb-4bde-965d-005f7291b273 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:51.860897 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1048/2094] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:17:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:17:52.127623 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27633e0a-6be1-4eea-9d07-be03172f649d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:52.129066 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-27633e0a-6be1-4eea-9d07-be03172f649d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:52.142017 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27633e0a-6be1-4eea-9d07-be03172f649d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:52.146731 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a18258-da5c-4c1b-8794-354825dbacfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:52.147404 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a18258-da5c-4c1b-8794-354825dbacfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:52.147684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a18258-da5c-4c1b-8794-354825dbacfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:52.148021 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a18258-da5c-4c1b-8794-354825dbacfe admin admin] PATH_INFO: `/v3/projects/bc61eb62575941baa41b079f1a697971` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:52.149165 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a18258-da5c-4c1b-8794-354825dbacfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc61eb62575941baa41b079f1a697971)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:52.151814 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a18258-da5c-4c1b-8794-354825dbacfe admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:52.207943 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e5a18258-da5c-4c1b-8794-354825dbacfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc61eb62575941baa41b079f1a697971', 'request_id': 'req-e5a18258-da5c-4c1b-8794-354825dbacfe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:52.214186 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e5a18258-da5c-4c1b-8794-354825dbacfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc61eb62575941baa41b079f1a697971', 'request_id': 'req-e5a18258-da5c-4c1b-8794-354825dbacfe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:52.220782 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e5a18258-da5c-4c1b-8794-354825dbacfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc61eb62575941baa41b079f1a697971', 'request_id': 'req-e5a18258-da5c-4c1b-8794-354825dbacfe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:52.235345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e5a18258-da5c-4c1b-8794-354825dbacfe admin admin] The token cache is being invalidate because project bc61eb62575941baa41b079f1a697971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:17:52.235642 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e5a18258-da5c-4c1b-8794-354825dbacfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:52.236342 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e5a18258-da5c-4c1b-8794-354825dbacfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:52.237888 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1047/2095] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:17:52 2022] DELETE /identity/v3/projects/bc61eb62575941baa41b079f1a697971 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:52.242461 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1bf9f-25cb-4bde-965d-005f7291b273 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:52.252127 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1bf9f-25cb-4bde-965d-005f7291b273 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:52.272366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:52.315873 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4dfe83-e6e6-4b66-a564-758bf5616be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:52.316619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4dfe83-e6e6-4b66-a564-758bf5616be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:52.317056 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4dfe83-e6e6-4b66-a564-758bf5616be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:52.317501 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4dfe83-e6e6-4b66-a564-758bf5616be7 admin admin] PATH_INFO: `/v3/users/66d56f4d800445d193c7c0dd62fe87ac` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:52.318791 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4dfe83-e6e6-4b66-a564-758bf5616be7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66d56f4d800445d193c7c0dd62fe87ac)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:52.321235 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4dfe83-e6e6-4b66-a564-758bf5616be7 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:52.369364 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-af4dfe83-e6e6-4b66-a564-758bf5616be7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66d56f4d800445d193c7c0dd62fe87ac'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:52.376272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-af4dfe83-e6e6-4b66-a564-758bf5616be7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66d56f4d800445d193c7c0dd62fe87ac'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:52.376928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-af4dfe83-e6e6-4b66-a564-758bf5616be7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66d56f4d800445d193c7c0dd62fe87ac'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:52.384438 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-af4dfe83-e6e6-4b66-a564-758bf5616be7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66d56f4d800445d193c7c0dd62fe87ac'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:52.396143 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-af4dfe83-e6e6-4b66-a564-758bf5616be7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66d56f4d800445d193c7c0dd62fe87ac'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:52.402407 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1049/2096] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:17:52 2022] DELETE /identity/v3/users/66d56f4d800445d193c7c0dd62fe87ac => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:52.666504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a18258-da5c-4c1b-8794-354825dbacfe admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:52.668001 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e5a18258-da5c-4c1b-8794-354825dbacfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:52.682519 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a18258-da5c-4c1b-8794-354825dbacfe admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:52.740136 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5418c0-b6fe-4158-91e3-85ba995e8f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:52.740790 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5418c0-b6fe-4158-91e3-85ba995e8f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:52.741203 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5418c0-b6fe-4158-91e3-85ba995e8f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:52.741776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5418c0-b6fe-4158-91e3-85ba995e8f23 admin admin] PATH_INFO: `/v3/projects/16d88cf51228474c84ad9d495494115c` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:52.742646 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5418c0-b6fe-4158-91e3-85ba995e8f23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16d88cf51228474c84ad9d495494115c)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:52.746438 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5418c0-b6fe-4158-91e3-85ba995e8f23 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:52.780684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:52.806345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7b5418c0-b6fe-4158-91e3-85ba995e8f23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16d88cf51228474c84ad9d495494115c', 'request_id': 'req-7b5418c0-b6fe-4158-91e3-85ba995e8f23'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:52.812190 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7b5418c0-b6fe-4158-91e3-85ba995e8f23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16d88cf51228474c84ad9d495494115c', 'request_id': 'req-7b5418c0-b6fe-4158-91e3-85ba995e8f23'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:52.818610 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7b5418c0-b6fe-4158-91e3-85ba995e8f23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16d88cf51228474c84ad9d495494115c', 'request_id': 'req-7b5418c0-b6fe-4158-91e3-85ba995e8f23'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:52.832677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7b5418c0-b6fe-4158-91e3-85ba995e8f23 admin admin] The token cache is being invalidate because project 16d88cf51228474c84ad9d495494115c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:17:52.833021 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7b5418c0-b6fe-4158-91e3-85ba995e8f23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:52.833680 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7b5418c0-b6fe-4158-91e3-85ba995e8f23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:52.835429 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1048/2097] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:17:52 2022] DELETE /identity/v3/projects/16d88cf51228474c84ad9d495494115c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:52.841214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f57e3-d2a2-4bbb-8ac4-5a39b5fa7f79 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:52.841499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f57e3-d2a2-4bbb-8ac4-5a39b5fa7f79 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:52.841936 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f57e3-d2a2-4bbb-8ac4-5a39b5fa7f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:52.860395 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5418c0-b6fe-4158-91e3-85ba995e8f23 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:52.861871 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7b5418c0-b6fe-4158-91e3-85ba995e8f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:52.866211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b32f57e3-d2a2-4bbb-8ac4-5a39b5fa7f79 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:52.867625 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b32f57e3-d2a2-4bbb-8ac4-5a39b5fa7f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:52.934108 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1050/2098] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:17:52.943138 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70dfb4-26ca-4f5b-bd44-837e02ebf504 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:52.943493 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70dfb4-26ca-4f5b-bd44-837e02ebf504 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:52.943771 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70dfb4-26ca-4f5b-bd44-837e02ebf504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:52.947954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5418c0-b6fe-4158-91e3-85ba995e8f23 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:52.953655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fad85-470d-40cd-a5a4-3b701fde7d96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:52.954273 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fad85-470d-40cd-a5a4-3b701fde7d96 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:52.954603 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fad85-470d-40cd-a5a4-3b701fde7d96 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:52.954933 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fad85-470d-40cd-a5a4-3b701fde7d96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:52.956845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f25fad85-470d-40cd-a5a4-3b701fde7d96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:52.964060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-1d70dfb4-26ca-4f5b-bd44-837e02ebf504 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:52.965262 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1d70dfb4-26ca-4f5b-bd44-837e02ebf504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:52.969621 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25fad85-470d-40cd-a5a4-3b701fde7d96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:52.971225 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25fad85-470d-40cd-a5a4-3b701fde7d96 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:52.973430 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1051/2099] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:17:52.977700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70dfb4-26ca-4f5b-bd44-837e02ebf504 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:52.982105 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1049/2100] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:17:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:17:52.987048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70dfb4-26ca-4f5b-bd44-837e02ebf504 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:52.992346 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528518f4-e205-4196-8b09-da8593aec709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:52.993529 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528518f4-e205-4196-8b09-da8593aec709 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:52.994008 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528518f4-e205-4196-8b09-da8593aec709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:52.994348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528518f4-e205-4196-8b09-da8593aec709 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:52.995102 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528518f4-e205-4196-8b09-da8593aec709 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:17:52.995501 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528518f4-e205-4196-8b09-da8593aec709 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:52.997156 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528518f4-e205-4196-8b09-da8593aec709 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:53.009700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1052/2101] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:17:53.014210 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fad85-470d-40cd-a5a4-3b701fde7d96 service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:53.025410 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fad85-470d-40cd-a5a4-3b701fde7d96 service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:53.030070 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50681c7-1c53-46b0-a754-205fcf46562a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:53.030674 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50681c7-1c53-46b0-a754-205fcf46562a admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:53.030982 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50681c7-1c53-46b0-a754-205fcf46562a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:53.031286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50681c7-1c53-46b0-a754-205fcf46562a admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:53.031930 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50681c7-1c53-46b0-a754-205fcf46562a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:53.033822 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50681c7-1c53-46b0-a754-205fcf46562a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:53.061126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1050/2102] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:17:53 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:17:53.065317 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528518f4-e205-4196-8b09-da8593aec709 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:53.075948 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528518f4-e205-4196-8b09-da8593aec709 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:53.094054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:53.132249 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76537783-0631-4bbd-9ac4-e08997f3ccd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:53.132841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76537783-0631-4bbd-9ac4-e08997f3ccd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:53.133117 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76537783-0631-4bbd-9ac4-e08997f3ccd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:53.133387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76537783-0631-4bbd-9ac4-e08997f3ccd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:53.134798 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76537783-0631-4bbd-9ac4-e08997f3ccd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:53.136730 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76537783-0631-4bbd-9ac4-e08997f3ccd5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:53.156464 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1053/2103] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:17:53 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:17:53.161052 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50681c7-1c53-46b0-a754-205fcf46562a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:53.171293 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50681c7-1c53-46b0-a754-205fcf46562a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:53.176804 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f741b2d-6ef7-4e5f-a04d-da9ae430e0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:53.177500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f741b2d-6ef7-4e5f-a04d-da9ae430e0de admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:53.177849 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f741b2d-6ef7-4e5f-a04d-da9ae430e0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:53.178119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f741b2d-6ef7-4e5f-a04d-da9ae430e0de admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:53.178904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f741b2d-6ef7-4e5f-a04d-da9ae430e0de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:53.181209 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f741b2d-6ef7-4e5f-a04d-da9ae430e0de admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:53.191715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1051/2104] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:17:53.195332 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76537783-0631-4bbd-9ac4-e08997f3ccd5 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:53.206598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76537783-0631-4bbd-9ac4-e08997f3ccd5 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:53.211245 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9485f-4a9e-4101-b6b3-26520697ef05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:53.211843 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9485f-4a9e-4101-b6b3-26520697ef05 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:53.212128 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9485f-4a9e-4101-b6b3-26520697ef05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:53.212425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9485f-4a9e-4101-b6b3-26520697ef05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:53.213106 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9485f-4a9e-4101-b6b3-26520697ef05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:53.214710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9485f-4a9e-4101-b6b3-26520697ef05 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:53.225439 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1054/2105] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:17:53.230204 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f741b2d-6ef7-4e5f-a04d-da9ae430e0de admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:53.240217 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f741b2d-6ef7-4e5f-a04d-da9ae430e0de admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:53.245666 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8e31b-b4c3-42d2-a3f3-739899e64206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:53.246319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8e31b-b4c3-42d2-a3f3-739899e64206 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:53.246604 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8e31b-b4c3-42d2-a3f3-739899e64206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:53.246892 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8e31b-b4c3-42d2-a3f3-739899e64206 admin admin] PATH_INFO: `/v3/projects/ff859f5c6d0c46d38ee3a9ec52345005/users/f8413236637e422da8ff4357b50369a5/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:53.262705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8e31b-b4c3-42d2-a3f3-739899e64206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff859f5c6d0c46d38ee3a9ec52345005, user_id=f8413236637e422da8ff4357b50369a5, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:53.265386 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8e31b-b4c3-42d2-a3f3-739899e64206 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:53.278079 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1052/2106] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:17:53 2022] PUT /identity/v3/projects/ff859f5c6d0c46d38ee3a9ec52345005/users/f8413236637e422da8ff4357b50369a5/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:53.288645 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55369d48-41b0-44a1-a07e-c2e12412fcd0 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:53.289105 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55369d48-41b0-44a1-a07e-c2e12412fcd0 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:53.289208 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55369d48-41b0-44a1-a07e-c2e12412fcd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:53.317769 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-55369d48-41b0-44a1-a07e-c2e12412fcd0 None None] MFA Rules not processed for user `f8413236637e422da8ff4357b50369a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:53.319058 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-55369d48-41b0-44a1-a07e-c2e12412fcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:53.384272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1055/2107] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:17:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:17:53.388766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8e31b-b4c3-42d2-a3f3-739899e64206 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:53.434732 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:53.457463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8e31b-b4c3-42d2-a3f3-739899e64206 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:53.462917 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fb7e7-bc47-441a-8f18-3ac6a07d7bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:53.463689 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fb7e7-bc47-441a-8f18-3ac6a07d7bdc admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:53.463974 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fb7e7-bc47-441a-8f18-3ac6a07d7bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:53.464685 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fb7e7-bc47-441a-8f18-3ac6a07d7bdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:53.464958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fb7e7-bc47-441a-8f18-3ac6a07d7bdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:53.466897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fb7e7-bc47-441a-8f18-3ac6a07d7bdc admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:53.507161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1053/2108] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:17:53 2022] POST /identity/v3/projects => generated 367 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:17:53.512365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55369d48-41b0-44a1-a07e-c2e12412fcd0 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:53.523728 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55369d48-41b0-44a1-a07e-c2e12412fcd0 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:53.581827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22bd11-3edb-42ff-814a-e10e0f9613a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:53.582433 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22bd11-3edb-42ff-814a-e10e0f9613a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:53.582711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22bd11-3edb-42ff-814a-e10e0f9613a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:53.583014 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22bd11-3edb-42ff-814a-e10e0f9613a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:53.583668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22bd11-3edb-42ff-814a-e10e0f9613a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:53.585641 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22bd11-3edb-42ff-814a-e10e0f9613a3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:53.606813 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1056/2109] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:17:53 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:17:53.610910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fb7e7-bc47-441a-8f18-3ac6a07d7bdc admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:53.621345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fb7e7-bc47-441a-8f18-3ac6a07d7bdc admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:53.625930 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2a521-cbf7-4dec-bd20-bf71bb27eeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:53.626521 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2a521-cbf7-4dec-bd20-bf71bb27eeb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:53.626799 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2a521-cbf7-4dec-bd20-bf71bb27eeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:53.627100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2a521-cbf7-4dec-bd20-bf71bb27eeb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:53.627775 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2a521-cbf7-4dec-bd20-bf71bb27eeb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:53.629349 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2a521-cbf7-4dec-bd20-bf71bb27eeb4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:53.640205 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1054/2110] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:17:53.644266 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22bd11-3edb-42ff-814a-e10e0f9613a3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:53.654360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22bd11-3edb-42ff-814a-e10e0f9613a3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:53.658969 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c9e00-c1ee-48fd-96f5-eb4089ce779b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:53.659615 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c9e00-c1ee-48fd-96f5-eb4089ce779b admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:53.660151 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c9e00-c1ee-48fd-96f5-eb4089ce779b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:53.660430 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c9e00-c1ee-48fd-96f5-eb4089ce779b admin admin] PATH_INFO: `/v3/projects/d4b687ba27ce49dab0b8faf881880aac/users/d0528b667bf346c8964f1a5addf747d9/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:53.675348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c9e00-c1ee-48fd-96f5-eb4089ce779b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4b687ba27ce49dab0b8faf881880aac, user_id=d0528b667bf346c8964f1a5addf747d9, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:53.678085 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c9e00-c1ee-48fd-96f5-eb4089ce779b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:53.681943 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:53.692604 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1057/2111] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:17:53 2022] PUT /identity/v3/projects/d4b687ba27ce49dab0b8faf881880aac/users/d0528b667bf346c8964f1a5addf747d9/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:53.697164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2a521-cbf7-4dec-bd20-bf71bb27eeb4 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:53.706889 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2a521-cbf7-4dec-bd20-bf71bb27eeb4 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:53.762633 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedb1c9-3496-451e-bba5-f84dcb9a0209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:53.763283 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedb1c9-3496-451e-bba5-f84dcb9a0209 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:53.763568 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedb1c9-3496-451e-bba5-f84dcb9a0209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:53.763843 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedb1c9-3496-451e-bba5-f84dcb9a0209 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:53.764479 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedb1c9-3496-451e-bba5-f84dcb9a0209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:53.766055 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedb1c9-3496-451e-bba5-f84dcb9a0209 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:53.775961 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1055/2112] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:17:53.780128 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c9e00-c1ee-48fd-96f5-eb4089ce779b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:53.790809 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c9e00-c1ee-48fd-96f5-eb4089ce779b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:53.795689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed7799-b92c-428d-99e5-72438a16a11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:53.797333 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed7799-b92c-428d-99e5-72438a16a11a admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:53.797333 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed7799-b92c-428d-99e5-72438a16a11a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:53.797333 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed7799-b92c-428d-99e5-72438a16a11a admin admin] PATH_INFO: `/v3/projects/d4b687ba27ce49dab0b8faf881880aac/users/d0528b667bf346c8964f1a5addf747d9/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:53.798827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed7799-b92c-428d-99e5-72438a16a11a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4b687ba27ce49dab0b8faf881880aac, user_id=d0528b667bf346c8964f1a5addf747d9, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:53.801513 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed7799-b92c-428d-99e5-72438a16a11a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:53.815417 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1058/2113] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:17:53 2022] PUT /identity/v3/projects/d4b687ba27ce49dab0b8faf881880aac/users/d0528b667bf346c8964f1a5addf747d9/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:53.828877 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be95fe1-93be-4876-8a2f-d186a8d92b0c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:53.829409 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be95fe1-93be-4876-8a2f-d186a8d92b0c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:53.829770 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be95fe1-93be-4876-8a2f-d186a8d92b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:53.864219 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-5be95fe1-93be-4876-8a2f-d186a8d92b0c None None] MFA Rules not processed for user `d0528b667bf346c8964f1a5addf747d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:53.865554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5be95fe1-93be-4876-8a2f-d186a8d92b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:53.947004 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1056/2114] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:17:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:17:53.958451 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed7799-b92c-428d-99e5-72438a16a11a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:53.960237 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-06ed7799-b92c-428d-99e5-72438a16a11a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:54.016660 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed7799-b92c-428d-99e5-72438a16a11a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:54.020283 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937597b-80ab-459a-b11c-c6213675b35b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:54.020875 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937597b-80ab-459a-b11c-c6213675b35b service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:54.021150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937597b-80ab-459a-b11c-c6213675b35b service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:54.021419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937597b-80ab-459a-b11c-c6213675b35b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:54.030197 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937597b-80ab-459a-b11c-c6213675b35b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:54.031653 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937597b-80ab-459a-b11c-c6213675b35b service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:54.041838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1059/2115] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:17:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:17:54.087104 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be95fe1-93be-4876-8a2f-d186a8d92b0c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:54.088558 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5be95fe1-93be-4876-8a2f-d186a8d92b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:54.100564 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be95fe1-93be-4876-8a2f-d186a8d92b0c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:54.104811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ef088-46d4-4f53-9ea0-762591da9c9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:54.105411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ef088-46d4-4f53-9ea0-762591da9c9f service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:54.105691 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ef088-46d4-4f53-9ea0-762591da9c9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:54.105966 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ef088-46d4-4f53-9ea0-762591da9c9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:54.138628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:54.163143 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ef088-46d4-4f53-9ea0-762591da9c9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:54.164583 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ef088-46d4-4f53-9ea0-762591da9c9f service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:54.174247 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1057/2116] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:17:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:17:55.031764 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703005fb-d341-4793-aedc-6084860722a7 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:55.032244 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703005fb-d341-4793-aedc-6084860722a7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:55.032458 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703005fb-d341-4793-aedc-6084860722a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:55.052335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-703005fb-d341-4793-aedc-6084860722a7 None None] MFA Rules not processed for user `280d47ed79f14105a83b18b2cac58b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:55.053538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-703005fb-d341-4793-aedc-6084860722a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:55.113523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1060/2117] 10.222.0.27 () {58 vars in 967 bytes} [Mon Oct 17 03:17:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:17:55.126317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ef088-46d4-4f53-9ea0-762591da9c9f service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:55.127748 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7e5ef088-46d4-4f53-9ea0-762591da9c9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:55.198739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ef088-46d4-4f53-9ea0-762591da9c9f service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:55.203941 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117ceef-d982-4195-8fba-87d62a9ab821 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eceda8b3a8ba4233851d0138d8f4d06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:55.204557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117ceef-d982-4195-8fba-87d62a9ab821 service swift] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:55.204840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117ceef-d982-4195-8fba-87d62a9ab821 service swift] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:55.205113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117ceef-d982-4195-8fba-87d62a9ab821 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:55.213110 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117ceef-d982-4195-8fba-87d62a9ab821 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:55.214691 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117ceef-d982-4195-8fba-87d62a9ab821 service swift] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:55.223136 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1058/2118] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Oct 17 03:17:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:17:57.705026 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703005fb-d341-4793-aedc-6084860722a7 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:57.767832 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703005fb-d341-4793-aedc-6084860722a7 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:57.772306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f628b2e-d29a-4f65-9fee-9e76f6284e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:57.772929 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f628b2e-d29a-4f65-9fee-9e76f6284e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:57.773211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f628b2e-d29a-4f65-9fee-9e76f6284e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:57.773486 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f628b2e-d29a-4f65-9fee-9e76f6284e96 admin admin] PATH_INFO: `/v3/users/f8413236637e422da8ff4357b50369a5` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:57.775482 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f628b2e-d29a-4f65-9fee-9e76f6284e96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8413236637e422da8ff4357b50369a5)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:57.777433 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f628b2e-d29a-4f65-9fee-9e76f6284e96 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:57.810108 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:57.824279 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5f628b2e-d29a-4f65-9fee-9e76f6284e96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8413236637e422da8ff4357b50369a5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:57.830952 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5f628b2e-d29a-4f65-9fee-9e76f6284e96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8413236637e422da8ff4357b50369a5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:57.831606 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5f628b2e-d29a-4f65-9fee-9e76f6284e96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8413236637e422da8ff4357b50369a5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:57.840457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5f628b2e-d29a-4f65-9fee-9e76f6284e96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8413236637e422da8ff4357b50369a5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:57.854517 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5f628b2e-d29a-4f65-9fee-9e76f6284e96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8413236637e422da8ff4357b50369a5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:57.860593 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1061/2119] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:17:57 2022] DELETE /identity/v3/users/f8413236637e422da8ff4357b50369a5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:57.865121 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4325e51-f16d-4189-9cc8-5c56b4fcd970 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:57.865602 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4325e51-f16d-4189-9cc8-5c56b4fcd970 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:57.865700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4325e51-f16d-4189-9cc8-5c56b4fcd970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:57.884822 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-d4325e51-f16d-4189-9cc8-5c56b4fcd970 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:57.886014 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d4325e51-f16d-4189-9cc8-5c56b4fcd970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:57.944346 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1059/2120] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:17:57.954302 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f628b2e-d29a-4f65-9fee-9e76f6284e96 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:57.955723 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5f628b2e-d29a-4f65-9fee-9e76f6284e96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:58.013725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f628b2e-d29a-4f65-9fee-9e76f6284e96 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:58.017357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71ec0c-dcdf-4679-bef4-915176dab30a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:58.017961 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71ec0c-dcdf-4679-bef4-915176dab30a service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:58.018241 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71ec0c-dcdf-4679-bef4-915176dab30a service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:58.018513 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71ec0c-dcdf-4679-bef4-915176dab30a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:58.028150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71ec0c-dcdf-4679-bef4-915176dab30a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:58.029580 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71ec0c-dcdf-4679-bef4-915176dab30a service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:58.039105 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1062/2121] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:17:58.298640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4325e51-f16d-4189-9cc8-5c56b4fcd970 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:58.300265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d4325e51-f16d-4189-9cc8-5c56b4fcd970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:58.313404 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4325e51-f16d-4189-9cc8-5c56b4fcd970 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:58.317992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6bc08-fc05-4bf6-8e8e-624a50054977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:58.318623 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6bc08-fc05-4bf6-8e8e-624a50054977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:58.318938 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6bc08-fc05-4bf6-8e8e-624a50054977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:58.319218 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6bc08-fc05-4bf6-8e8e-624a50054977 admin admin] PATH_INFO: `/v3/projects/ff859f5c6d0c46d38ee3a9ec52345005` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:58.320341 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a6bc08-fc05-4bf6-8e8e-624a50054977 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff859f5c6d0c46d38ee3a9ec52345005)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:58.323376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a6bc08-fc05-4bf6-8e8e-624a50054977 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:58.370688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-80a6bc08-fc05-4bf6-8e8e-624a50054977 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff859f5c6d0c46d38ee3a9ec52345005', 'request_id': 'req-80a6bc08-fc05-4bf6-8e8e-624a50054977'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:58.376825 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-80a6bc08-fc05-4bf6-8e8e-624a50054977 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff859f5c6d0c46d38ee3a9ec52345005', 'request_id': 'req-80a6bc08-fc05-4bf6-8e8e-624a50054977'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:58.379045 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:58.385324 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-80a6bc08-fc05-4bf6-8e8e-624a50054977 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff859f5c6d0c46d38ee3a9ec52345005', 'request_id': 'req-80a6bc08-fc05-4bf6-8e8e-624a50054977'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:58.406512 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-80a6bc08-fc05-4bf6-8e8e-624a50054977 admin admin] The token cache is being invalidate because project ff859f5c6d0c46d38ee3a9ec52345005 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:17:58.406512 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-80a6bc08-fc05-4bf6-8e8e-624a50054977 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:58.406512 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-80a6bc08-fc05-4bf6-8e8e-624a50054977 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:58.406512 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1060/2122] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:17:58 2022] DELETE /identity/v3/projects/ff859f5c6d0c46d38ee3a9ec52345005 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:58.406880 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71ec0c-dcdf-4679-bef4-915176dab30a service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:58.416241 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71ec0c-dcdf-4679-bef4-915176dab30a service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:58.472528 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b05420-2353-434e-9629-d211e1a91701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:58.473148 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b05420-2353-434e-9629-d211e1a91701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:58.473425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b05420-2353-434e-9629-d211e1a91701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:58.473700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b05420-2353-434e-9629-d211e1a91701 admin admin] PATH_INFO: `/v3/users/d0528b667bf346c8964f1a5addf747d9` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:58.474801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b05420-2353-434e-9629-d211e1a91701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0528b667bf346c8964f1a5addf747d9)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:58.476738 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b05420-2353-434e-9629-d211e1a91701 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:58.521746 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-04b05420-2353-434e-9629-d211e1a91701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0528b667bf346c8964f1a5addf747d9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:58.526980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:58.529193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-04b05420-2353-434e-9629-d211e1a91701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0528b667bf346c8964f1a5addf747d9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:58.529949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-04b05420-2353-434e-9629-d211e1a91701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0528b667bf346c8964f1a5addf747d9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:58.538488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-04b05420-2353-434e-9629-d211e1a91701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0528b667bf346c8964f1a5addf747d9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:58.551238 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-04b05420-2353-434e-9629-d211e1a91701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0528b667bf346c8964f1a5addf747d9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:58.557710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1063/2123] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:17:58 2022] DELETE /identity/v3/users/d0528b667bf346c8964f1a5addf747d9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:58.825696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6bc08-fc05-4bf6-8e8e-624a50054977 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:58.827133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-80a6bc08-fc05-4bf6-8e8e-624a50054977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:58.842748 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6bc08-fc05-4bf6-8e8e-624a50054977 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:58.900227 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1e932-fea6-425d-b8b2-dfb856360f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:58.900951 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1e932-fea6-425d-b8b2-dfb856360f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:58.901234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1e932-fea6-425d-b8b2-dfb856360f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:58.901510 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1e932-fea6-425d-b8b2-dfb856360f39 admin admin] PATH_INFO: `/v3/projects/d4b687ba27ce49dab0b8faf881880aac` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:58.902628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1e932-fea6-425d-b8b2-dfb856360f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4b687ba27ce49dab0b8faf881880aac)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:58.904590 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1e932-fea6-425d-b8b2-dfb856360f39 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:58.952047 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fad1e932-fea6-425d-b8b2-dfb856360f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4b687ba27ce49dab0b8faf881880aac', 'request_id': 'req-fad1e932-fea6-425d-b8b2-dfb856360f39'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:58.957505 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fad1e932-fea6-425d-b8b2-dfb856360f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4b687ba27ce49dab0b8faf881880aac', 'request_id': 'req-fad1e932-fea6-425d-b8b2-dfb856360f39'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:58.964536 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fad1e932-fea6-425d-b8b2-dfb856360f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4b687ba27ce49dab0b8faf881880aac', 'request_id': 'req-fad1e932-fea6-425d-b8b2-dfb856360f39'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:58.976773 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fad1e932-fea6-425d-b8b2-dfb856360f39 admin admin] The token cache is being invalidate because project d4b687ba27ce49dab0b8faf881880aac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:17:58.977063 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fad1e932-fea6-425d-b8b2-dfb856360f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:58.978217 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fad1e932-fea6-425d-b8b2-dfb856360f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:17:58.979740 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1061/2124] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:17:58 2022] DELETE /identity/v3/projects/d4b687ba27ce49dab0b8faf881880aac => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:58.984499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725db0c2-fac6-4d27-ba89-c09a37a674c2 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:58.984987 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725db0c2-fac6-4d27-ba89-c09a37a674c2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:58.985103 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725db0c2-fac6-4d27-ba89-c09a37a674c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:59.009739 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-725db0c2-fac6-4d27-ba89-c09a37a674c2 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:59.011051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-725db0c2-fac6-4d27-ba89-c09a37a674c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:59.069780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1064/2125] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:17:59.078447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae56b50-4ecf-4a1c-bb04-bc92f6db37a8 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:59.078758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae56b50-4ecf-4a1c-bb04-bc92f6db37a8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:59.079077 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae56b50-4ecf-4a1c-bb04-bc92f6db37a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:59.099189 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4ae56b50-4ecf-4a1c-bb04-bc92f6db37a8 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:59.100396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4ae56b50-4ecf-4a1c-bb04-bc92f6db37a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:59.108785 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1062/2126] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:17:59.112846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725db0c2-fac6-4d27-ba89-c09a37a674c2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:59.122665 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725db0c2-fac6-4d27-ba89-c09a37a674c2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:59.127284 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4b6fc-1ff7-4c1d-be75-dcbef1c2e28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:59.127879 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4b6fc-1ff7-4c1d-be75-dcbef1c2e28f admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:59.128158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4b6fc-1ff7-4c1d-be75-dcbef1c2e28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:59.128425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4b6fc-1ff7-4c1d-be75-dcbef1c2e28f admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:59.129066 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4b6fc-1ff7-4c1d-be75-dcbef1c2e28f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:17:59.129365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4b6fc-1ff7-4c1d-be75-dcbef1c2e28f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:59.130973 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4b6fc-1ff7-4c1d-be75-dcbef1c2e28f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:59.142255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1065/2127] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:17:59.146477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae56b50-4ecf-4a1c-bb04-bc92f6db37a8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:59.156273 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae56b50-4ecf-4a1c-bb04-bc92f6db37a8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:59.161070 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf3230-963d-49e5-ba79-7f5bf0322e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:59.161659 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf3230-963d-49e5-ba79-7f5bf0322e6a admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:59.161942 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf3230-963d-49e5-ba79-7f5bf0322e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:59.162239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf3230-963d-49e5-ba79-7f5bf0322e6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:59.162891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf3230-963d-49e5-ba79-7f5bf0322e6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:59.164823 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf3230-963d-49e5-ba79-7f5bf0322e6a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:59.191512 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1063/2128] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:17:59 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:17:59.195747 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4b6fc-1ff7-4c1d-be75-dcbef1c2e28f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:59.205803 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4b6fc-1ff7-4c1d-be75-dcbef1c2e28f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:59.261854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b2501-d1b2-45bc-92a6-031aa95d6f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:59.262454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b2501-d1b2-45bc-92a6-031aa95d6f85 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:59.262731 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b2501-d1b2-45bc-92a6-031aa95d6f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:59.263039 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b2501-d1b2-45bc-92a6-031aa95d6f85 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:59.263672 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b2501-d1b2-45bc-92a6-031aa95d6f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:59.265581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b2501-d1b2-45bc-92a6-031aa95d6f85 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:59.289308 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1066/2129] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:17:59 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:17:59.293263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf3230-963d-49e5-ba79-7f5bf0322e6a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:59.303382 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf3230-963d-49e5-ba79-7f5bf0322e6a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:59.308168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b7914-eb54-4c69-b197-5270099f81c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:59.308766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b7914-eb54-4c69-b197-5270099f81c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:59.309050 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b7914-eb54-4c69-b197-5270099f81c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:59.309323 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b7914-eb54-4c69-b197-5270099f81c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:59.309965 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b7914-eb54-4c69-b197-5270099f81c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:59.311608 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b7914-eb54-4c69-b197-5270099f81c5 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:59.321884 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1064/2130] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:17:59.325698 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b2501-d1b2-45bc-92a6-031aa95d6f85 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:59.335355 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b2501-d1b2-45bc-92a6-031aa95d6f85 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:59.340465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139771ae-52ba-4e7d-98f7-883e9e3ccdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:59.341049 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139771ae-52ba-4e7d-98f7-883e9e3ccdca admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:59.341326 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139771ae-52ba-4e7d-98f7-883e9e3ccdca admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:59.341599 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139771ae-52ba-4e7d-98f7-883e9e3ccdca admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:59.342261 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139771ae-52ba-4e7d-98f7-883e9e3ccdca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:59.343960 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139771ae-52ba-4e7d-98f7-883e9e3ccdca admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:59.354590 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1067/2131] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:17:59.358632 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b7914-eb54-4c69-b197-5270099f81c5 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:59.368449 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b7914-eb54-4c69-b197-5270099f81c5 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:59.373230 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a73b5-de2b-4c62-9688-9c0b633b8a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:59.373875 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a73b5-de2b-4c62-9688-9c0b633b8a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:59.374155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a73b5-de2b-4c62-9688-9c0b633b8a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:59.374429 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a73b5-de2b-4c62-9688-9c0b633b8a96 admin admin] PATH_INFO: `/v3/projects/efc506bf933a4379b4831bbc26a70b85/users/7855d80d27964217a6b8e5fceb550b9f/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:59.389502 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a73b5-de2b-4c62-9688-9c0b633b8a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc506bf933a4379b4831bbc26a70b85, user_id=7855d80d27964217a6b8e5fceb550b9f, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:59.392173 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a73b5-de2b-4c62-9688-9c0b633b8a96 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:59.405477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1065/2132] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:17:59 2022] PUT /identity/v3/projects/efc506bf933a4379b4831bbc26a70b85/users/7855d80d27964217a6b8e5fceb550b9f/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:59.415981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb2b09-adab-474a-820b-ce68d48ab955 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:59.416289 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb2b09-adab-474a-820b-ce68d48ab955 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:59.416579 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb2b09-adab-474a-820b-ce68d48ab955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:59.444920 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a5fb2b09-adab-474a-820b-ce68d48ab955 None None] MFA Rules not processed for user `7855d80d27964217a6b8e5fceb550b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:59.446691 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a5fb2b09-adab-474a-820b-ce68d48ab955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:17:59.474291 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:59.515148 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1068/2133] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:17:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:17:59.519691 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a73b5-de2b-4c62-9688-9c0b633b8a96 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:59.537887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:17:59.583219 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a73b5-de2b-4c62-9688-9c0b633b8a96 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:59.588475 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b28ef-0505-40fd-ac5e-8d94b888a194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:59.589075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b28ef-0505-40fd-ac5e-8d94b888a194 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:59.589357 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b28ef-0505-40fd-ac5e-8d94b888a194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:59.589631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b28ef-0505-40fd-ac5e-8d94b888a194 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:59.590274 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b28ef-0505-40fd-ac5e-8d94b888a194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:59.592211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b28ef-0505-40fd-ac5e-8d94b888a194 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:59.617839 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1066/2134] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:17:59 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:17:59.622392 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb2b09-adab-474a-820b-ce68d48ab955 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:59.632911 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb2b09-adab-474a-820b-ce68d48ab955 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:59.692421 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece158f-1a2e-45b8-b3f8-cb8a7abc73c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:59.692953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece158f-1a2e-45b8-b3f8-cb8a7abc73c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:59.693235 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece158f-1a2e-45b8-b3f8-cb8a7abc73c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:59.693509 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece158f-1a2e-45b8-b3f8-cb8a7abc73c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:59.694142 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece158f-1a2e-45b8-b3f8-cb8a7abc73c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:59.696875 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece158f-1a2e-45b8-b3f8-cb8a7abc73c1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:59.717970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1069/2135] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:17:59 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:17:59.722027 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b28ef-0505-40fd-ac5e-8d94b888a194 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:59.732412 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b28ef-0505-40fd-ac5e-8d94b888a194 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:59.737043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92c7e8-e954-4177-b9f0-dc446857bb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:59.737642 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92c7e8-e954-4177-b9f0-dc446857bb47 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:59.737924 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92c7e8-e954-4177-b9f0-dc446857bb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:59.738198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92c7e8-e954-4177-b9f0-dc446857bb47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:59.738840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92c7e8-e954-4177-b9f0-dc446857bb47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:59.740509 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92c7e8-e954-4177-b9f0-dc446857bb47 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:59.751584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1067/2136] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:17:59.755548 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece158f-1a2e-45b8-b3f8-cb8a7abc73c1 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:59.765971 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece158f-1a2e-45b8-b3f8-cb8a7abc73c1 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:59.770714 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882201e4-890c-44b5-932a-829a1cc03c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:59.771465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882201e4-890c-44b5-932a-829a1cc03c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:59.771782 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882201e4-890c-44b5-932a-829a1cc03c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:59.772097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882201e4-890c-44b5-932a-829a1cc03c99 admin admin] PATH_INFO: `/v3/projects/44052dd57f9a4531814d3ccdd7105350/users/376c568c24c3483e96e47f2f5c40d59b/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:59.787047 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882201e4-890c-44b5-932a-829a1cc03c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44052dd57f9a4531814d3ccdd7105350, user_id=376c568c24c3483e96e47f2f5c40d59b, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:59.789711 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882201e4-890c-44b5-932a-829a1cc03c99 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:59.802370 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1070/2137] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:17:59 2022] PUT /identity/v3/projects/44052dd57f9a4531814d3ccdd7105350/users/376c568c24c3483e96e47f2f5c40d59b/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:59.806713 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92c7e8-e954-4177-b9f0-dc446857bb47 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:59.816521 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92c7e8-e954-4177-b9f0-dc446857bb47 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:59.872312 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bbff8-bfe2-4dca-8817-06e255143be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:59.872916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bbff8-bfe2-4dca-8817-06e255143be5 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:59.873194 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bbff8-bfe2-4dca-8817-06e255143be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:59.873464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bbff8-bfe2-4dca-8817-06e255143be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:59.874094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2bbff8-bfe2-4dca-8817-06e255143be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:59.875758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2bbff8-bfe2-4dca-8817-06e255143be5 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:59.885656 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1068/2138] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:17:59.889786 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882201e4-890c-44b5-932a-829a1cc03c99 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:17:59.900477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882201e4-890c-44b5-932a-829a1cc03c99 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:17:59.905089 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0725df-c26f-4704-b38d-20b30d60e78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:17:59.905736 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0725df-c26f-4704-b38d-20b30d60e78f admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:59.906019 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0725df-c26f-4704-b38d-20b30d60e78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:59.906297 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0725df-c26f-4704-b38d-20b30d60e78f admin admin] PATH_INFO: `/v3/projects/44052dd57f9a4531814d3ccdd7105350/users/376c568c24c3483e96e47f2f5c40d59b/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:59.908326 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0725df-c26f-4704-b38d-20b30d60e78f admin admin] RBAC: Authorizing `identity:create_grant(project_id=44052dd57f9a4531814d3ccdd7105350, user_id=376c568c24c3483e96e47f2f5c40d59b, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:17:59.911291 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0725df-c26f-4704-b38d-20b30d60e78f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:17:59.921927 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1071/2139] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:17:59 2022] PUT /identity/v3/projects/44052dd57f9a4531814d3ccdd7105350/users/376c568c24c3483e96e47f2f5c40d59b/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:17:59.932739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e959fe-9c64-412f-8d32-0c6b8275cb1c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:17:59.933242 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e959fe-9c64-412f-8d32-0c6b8275cb1c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:17:59.933351 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e959fe-9c64-412f-8d32-0c6b8275cb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:17:59.961000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-95e959fe-9c64-412f-8d32-0c6b8275cb1c None None] MFA Rules not processed for user `376c568c24c3483e96e47f2f5c40d59b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:17:59.962703 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-95e959fe-9c64-412f-8d32-0c6b8275cb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:00.033418 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1069/2140] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:17:59 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:00.044204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0725df-c26f-4704-b38d-20b30d60e78f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:00.045956 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6b0725df-c26f-4704-b38d-20b30d60e78f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:00.102853 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0725df-c26f-4704-b38d-20b30d60e78f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:00.106430 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10c849-d2cb-4577-b9ba-2d4986d35758 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:00.107059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10c849-d2cb-4577-b9ba-2d4986d35758 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:00.107335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10c849-d2cb-4577-b9ba-2d4986d35758 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:00.107603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10c849-d2cb-4577-b9ba-2d4986d35758 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:00.116357 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10c849-d2cb-4577-b9ba-2d4986d35758 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:00.117773 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10c849-d2cb-4577-b9ba-2d4986d35758 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:00.127254 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1072/2141] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:00.229207 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e959fe-9c64-412f-8d32-0c6b8275cb1c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:00.238924 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e959fe-9c64-412f-8d32-0c6b8275cb1c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:00.242453 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549de71-2fdd-4b5a-b8a1-489d4dac4ded service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:00.243086 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549de71-2fdd-4b5a-b8a1-489d4dac4ded service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:00.243363 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549de71-2fdd-4b5a-b8a1-489d4dac4ded service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:00.243958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549de71-2fdd-4b5a-b8a1-489d4dac4ded service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:00.251973 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:00.303776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6549de71-2fdd-4b5a-b8a1-489d4dac4ded service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:00.305203 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6549de71-2fdd-4b5a-b8a1-489d4dac4ded service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:00.314386 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1070/2142] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:02.898745 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10c849-d2cb-4577-b9ba-2d4986d35758 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:02.923096 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:02.960538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10c849-d2cb-4577-b9ba-2d4986d35758 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:02.964988 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425a8a8-8176-48a6-a326-11d6a1394ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:02.965603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425a8a8-8176-48a6-a326-11d6a1394ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:02.965880 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425a8a8-8176-48a6-a326-11d6a1394ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:02.966156 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425a8a8-8176-48a6-a326-11d6a1394ee1 admin admin] PATH_INFO: `/v3/users/7855d80d27964217a6b8e5fceb550b9f` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:02.967282 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e425a8a8-8176-48a6-a326-11d6a1394ee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7855d80d27964217a6b8e5fceb550b9f)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:02.969229 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e425a8a8-8176-48a6-a326-11d6a1394ee1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:03.016929 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e425a8a8-8176-48a6-a326-11d6a1394ee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7855d80d27964217a6b8e5fceb550b9f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:03.023786 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e425a8a8-8176-48a6-a326-11d6a1394ee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7855d80d27964217a6b8e5fceb550b9f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:03.024515 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e425a8a8-8176-48a6-a326-11d6a1394ee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7855d80d27964217a6b8e5fceb550b9f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:03.031510 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e425a8a8-8176-48a6-a326-11d6a1394ee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7855d80d27964217a6b8e5fceb550b9f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:03.044214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-e425a8a8-8176-48a6-a326-11d6a1394ee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7855d80d27964217a6b8e5fceb550b9f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:03.050303 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1073/2143] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:18:02 2022] DELETE /identity/v3/users/7855d80d27964217a6b8e5fceb550b9f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:03.054978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061a2dd-bd4a-4b14-b28e-f5e15d875cfd None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:03.055493 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061a2dd-bd4a-4b14-b28e-f5e15d875cfd None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:03.056148 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061a2dd-bd4a-4b14-b28e-f5e15d875cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:03.074929 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9061a2dd-bd4a-4b14-b28e-f5e15d875cfd None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:03.076135 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9061a2dd-bd4a-4b14-b28e-f5e15d875cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:03.134839 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1071/2144] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:03.144781 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425a8a8-8176-48a6-a326-11d6a1394ee1 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:03.146164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e425a8a8-8176-48a6-a326-11d6a1394ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:03.213219 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e425a8a8-8176-48a6-a326-11d6a1394ee1 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:03.216803 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2e176-dc55-49f0-b6d3-614fcf2f48e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:03.217407 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2e176-dc55-49f0-b6d3-614fcf2f48e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:03.217684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2e176-dc55-49f0-b6d3-614fcf2f48e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:03.217953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2e176-dc55-49f0-b6d3-614fcf2f48e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:03.227085 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2e176-dc55-49f0-b6d3-614fcf2f48e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:03.229009 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2e176-dc55-49f0-b6d3-614fcf2f48e0 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:03.238607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1074/2145] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:03.530542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061a2dd-bd4a-4b14-b28e-f5e15d875cfd None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:03.532004 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9061a2dd-bd4a-4b14-b28e-f5e15d875cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:03.544506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061a2dd-bd4a-4b14-b28e-f5e15d875cfd None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:03.549627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f4979-c527-4771-9f2a-a72929758db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:03.550266 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4979-c527-4771-9f2a-a72929758db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:03.550544 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4979-c527-4771-9f2a-a72929758db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:03.550814 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f4979-c527-4771-9f2a-a72929758db8 admin admin] PATH_INFO: `/v3/projects/efc506bf933a4379b4831bbc26a70b85` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:03.552237 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f4979-c527-4771-9f2a-a72929758db8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efc506bf933a4379b4831bbc26a70b85)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:03.554081 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f4979-c527-4771-9f2a-a72929758db8 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:03.600300 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-288f4979-c527-4771-9f2a-a72929758db8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efc506bf933a4379b4831bbc26a70b85', 'request_id': 'req-288f4979-c527-4771-9f2a-a72929758db8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:03.605648 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-288f4979-c527-4771-9f2a-a72929758db8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efc506bf933a4379b4831bbc26a70b85', 'request_id': 'req-288f4979-c527-4771-9f2a-a72929758db8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:03.611886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-288f4979-c527-4771-9f2a-a72929758db8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efc506bf933a4379b4831bbc26a70b85', 'request_id': 'req-288f4979-c527-4771-9f2a-a72929758db8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:03.624437 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-288f4979-c527-4771-9f2a-a72929758db8 admin admin] The token cache is being invalidate because project efc506bf933a4379b4831bbc26a70b85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:18:03.624728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-288f4979-c527-4771-9f2a-a72929758db8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:03.625358 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-288f4979-c527-4771-9f2a-a72929758db8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:03.626971 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1072/2146] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:18:03 2022] DELETE /identity/v3/projects/efc506bf933a4379b4831bbc26a70b85 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:03.630998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2e176-dc55-49f0-b6d3-614fcf2f48e0 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:03.641786 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2e176-dc55-49f0-b6d3-614fcf2f48e0 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:03.688937 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:03.707680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bfd18-ef08-4e85-81f6-77be800414f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:03.708355 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bfd18-ef08-4e85-81f6-77be800414f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:03.708636 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bfd18-ef08-4e85-81f6-77be800414f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:03.708908 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bfd18-ef08-4e85-81f6-77be800414f0 admin admin] PATH_INFO: `/v3/users/376c568c24c3483e96e47f2f5c40d59b` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:03.710080 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bfd18-ef08-4e85-81f6-77be800414f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=376c568c24c3483e96e47f2f5c40d59b)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:03.712360 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bfd18-ef08-4e85-81f6-77be800414f0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:03.759934 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e6bfd18-ef08-4e85-81f6-77be800414f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '376c568c24c3483e96e47f2f5c40d59b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:03.766690 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e6bfd18-ef08-4e85-81f6-77be800414f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '376c568c24c3483e96e47f2f5c40d59b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:03.767386 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e6bfd18-ef08-4e85-81f6-77be800414f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '376c568c24c3483e96e47f2f5c40d59b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:03.775263 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e6bfd18-ef08-4e85-81f6-77be800414f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '376c568c24c3483e96e47f2f5c40d59b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:03.787405 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e6bfd18-ef08-4e85-81f6-77be800414f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '376c568c24c3483e96e47f2f5c40d59b'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:03.793696 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1075/2147] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:18:03 2022] DELETE /identity/v3/users/376c568c24c3483e96e47f2f5c40d59b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:04.060339 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f4979-c527-4771-9f2a-a72929758db8 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:04.061865 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-288f4979-c527-4771-9f2a-a72929758db8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:04.076729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f4979-c527-4771-9f2a-a72929758db8 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:04.108475 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:04.136294 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0790a-c2f8-46f8-a63a-322d4fe05c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:04.136925 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0790a-c2f8-46f8-a63a-322d4fe05c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:04.137210 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0790a-c2f8-46f8-a63a-322d4fe05c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:04.137487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0790a-c2f8-46f8-a63a-322d4fe05c85 admin admin] PATH_INFO: `/v3/projects/44052dd57f9a4531814d3ccdd7105350` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:04.138614 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0790a-c2f8-46f8-a63a-322d4fe05c85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44052dd57f9a4531814d3ccdd7105350)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:04.140557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0790a-c2f8-46f8-a63a-322d4fe05c85 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:04.186063 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2ba0790a-c2f8-46f8-a63a-322d4fe05c85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44052dd57f9a4531814d3ccdd7105350', 'request_id': 'req-2ba0790a-c2f8-46f8-a63a-322d4fe05c85'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:04.191471 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2ba0790a-c2f8-46f8-a63a-322d4fe05c85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44052dd57f9a4531814d3ccdd7105350', 'request_id': 'req-2ba0790a-c2f8-46f8-a63a-322d4fe05c85'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:04.198162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2ba0790a-c2f8-46f8-a63a-322d4fe05c85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44052dd57f9a4531814d3ccdd7105350', 'request_id': 'req-2ba0790a-c2f8-46f8-a63a-322d4fe05c85'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:04.212840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2ba0790a-c2f8-46f8-a63a-322d4fe05c85 admin admin] The token cache is being invalidate because project 44052dd57f9a4531814d3ccdd7105350 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:18:04.213139 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2ba0790a-c2f8-46f8-a63a-322d4fe05c85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:04.213826 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2ba0790a-c2f8-46f8-a63a-322d4fe05c85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:04.215632 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1073/2148] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:18:04 2022] DELETE /identity/v3/projects/44052dd57f9a4531814d3ccdd7105350 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:04.220397 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dba5f-0f97-4eaf-a6f3-b591b949f128 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:04.220862 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dba5f-0f97-4eaf-a6f3-b591b949f128 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:04.220956 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0dba5f-0f97-4eaf-a6f3-b591b949f128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:04.243124 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-0c0dba5f-0f97-4eaf-a6f3-b591b949f128 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:04.244507 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0c0dba5f-0f97-4eaf-a6f3-b591b949f128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:04.308629 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1076/2149] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:04.316389 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59841569-c384-48e2-a5d2-3d7ef7deece9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:04.316676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59841569-c384-48e2-a5d2-3d7ef7deece9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:04.316952 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59841569-c384-48e2-a5d2-3d7ef7deece9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:04.337651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-59841569-c384-48e2-a5d2-3d7ef7deece9 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:04.339046 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-59841569-c384-48e2-a5d2-3d7ef7deece9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:04.347678 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1074/2150] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:04.352023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dba5f-0f97-4eaf-a6f3-b591b949f128 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:04.362683 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0dba5f-0f97-4eaf-a6f3-b591b949f128 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:04.367605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b3d64-203a-45fe-b7e3-ebcadc9b6516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:04.368317 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b3d64-203a-45fe-b7e3-ebcadc9b6516 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:04.368597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b3d64-203a-45fe-b7e3-ebcadc9b6516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:04.368868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b3d64-203a-45fe-b7e3-ebcadc9b6516 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:04.369515 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b3d64-203a-45fe-b7e3-ebcadc9b6516 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:18:04.369820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b3d64-203a-45fe-b7e3-ebcadc9b6516 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:04.371546 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b3d64-203a-45fe-b7e3-ebcadc9b6516 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:04.383410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1077/2151] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:18:04.387833 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59841569-c384-48e2-a5d2-3d7ef7deece9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:04.396976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59841569-c384-48e2-a5d2-3d7ef7deece9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:04.402198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e747f-3cea-4844-8c9b-8a8fa08713bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:04.402789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e747f-3cea-4844-8c9b-8a8fa08713bd admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:04.403102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e747f-3cea-4844-8c9b-8a8fa08713bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:04.403371 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e747f-3cea-4844-8c9b-8a8fa08713bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:04.403995 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e747f-3cea-4844-8c9b-8a8fa08713bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:04.405875 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e747f-3cea-4844-8c9b-8a8fa08713bd admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:04.433008 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1075/2152] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:18:04 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:04.437158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b3d64-203a-45fe-b7e3-ebcadc9b6516 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:04.446936 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b3d64-203a-45fe-b7e3-ebcadc9b6516 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:04.492564 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:04.504624 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542e2c6-e1d3-4dbb-b10b-b0609ea828bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:04.505238 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542e2c6-e1d3-4dbb-b10b-b0609ea828bf admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:04.505518 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542e2c6-e1d3-4dbb-b10b-b0609ea828bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:04.505791 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542e2c6-e1d3-4dbb-b10b-b0609ea828bf admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:04.506433 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542e2c6-e1d3-4dbb-b10b-b0609ea828bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:04.509207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542e2c6-e1d3-4dbb-b10b-b0609ea828bf admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:04.531396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1078/2153] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:18:04 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:04.535747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e747f-3cea-4844-8c9b-8a8fa08713bd admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:04.546285 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e747f-3cea-4844-8c9b-8a8fa08713bd admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:04.552441 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d8b56-faf8-4722-afcb-c7d78261ca86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:04.553148 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d8b56-faf8-4722-afcb-c7d78261ca86 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:04.553435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d8b56-faf8-4722-afcb-c7d78261ca86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:04.553710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d8b56-faf8-4722-afcb-c7d78261ca86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:04.554359 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d8b56-faf8-4722-afcb-c7d78261ca86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:04.556133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d8b56-faf8-4722-afcb-c7d78261ca86 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:04.569253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1076/2154] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:04.573960 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542e2c6-e1d3-4dbb-b10b-b0609ea828bf admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:04.588770 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542e2c6-e1d3-4dbb-b10b-b0609ea828bf admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:04.594050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7980f254-12d4-4189-b6d0-b140be0b2cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:04.594810 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980f254-12d4-4189-b6d0-b140be0b2cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:04.595172 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980f254-12d4-4189-b6d0-b140be0b2cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:04.595359 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980f254-12d4-4189-b6d0-b140be0b2cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:04.596015 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980f254-12d4-4189-b6d0-b140be0b2cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:04.597699 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7980f254-12d4-4189-b6d0-b140be0b2cc6 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:04.613968 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1079/2155] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:04.614984 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d8b56-faf8-4722-afcb-c7d78261ca86 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:04.625704 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d8b56-faf8-4722-afcb-c7d78261ca86 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:04.630342 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2897d9-5953-4f46-935c-f3df3519695d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:04.631077 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2897d9-5953-4f46-935c-f3df3519695d admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:04.631309 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2897d9-5953-4f46-935c-f3df3519695d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:04.631713 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2897d9-5953-4f46-935c-f3df3519695d admin admin] PATH_INFO: `/v3/projects/22430f2c851843e4a64cffb06adc3902/users/5c7b52cb14ba46e8a534cf038f3bbf3f/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:04.647396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2897d9-5953-4f46-935c-f3df3519695d admin admin] RBAC: Authorizing `identity:create_grant(project_id=22430f2c851843e4a64cffb06adc3902, user_id=5c7b52cb14ba46e8a534cf038f3bbf3f, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:04.650439 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2897d9-5953-4f46-935c-f3df3519695d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:04.663592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1077/2156] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:04 2022] PUT /identity/v3/projects/22430f2c851843e4a64cffb06adc3902/users/5c7b52cb14ba46e8a534cf038f3bbf3f/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:04.674642 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e20332-b13d-44ad-a9fb-5e64e9e978a0 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:04.675028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e20332-b13d-44ad-a9fb-5e64e9e978a0 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:04.675390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e20332-b13d-44ad-a9fb-5e64e9e978a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:04.704427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d3e20332-b13d-44ad-a9fb-5e64e9e978a0 None None] MFA Rules not processed for user `5c7b52cb14ba46e8a534cf038f3bbf3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:04.705568 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d3e20332-b13d-44ad-a9fb-5e64e9e978a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:04.798761 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1080/2157] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:04.811105 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2897d9-5953-4f46-935c-f3df3519695d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:04.873129 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2897d9-5953-4f46-935c-f3df3519695d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:04.877752 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce660211-24c0-4a41-8ab9-ee368527dceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:04.878350 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce660211-24c0-4a41-8ab9-ee368527dceb admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:04.878627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce660211-24c0-4a41-8ab9-ee368527dceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:04.878922 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce660211-24c0-4a41-8ab9-ee368527dceb admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:04.879555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce660211-24c0-4a41-8ab9-ee368527dceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:04.881433 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce660211-24c0-4a41-8ab9-ee368527dceb admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:04.886548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:04.910064 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1078/2158] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:18:04 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:18:04.914537 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e20332-b13d-44ad-a9fb-5e64e9e978a0 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:04.925504 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e20332-b13d-44ad-a9fb-5e64e9e978a0 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:04.981580 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe033d-5425-456f-a146-6b48c2fdd83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:04.982148 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe033d-5425-456f-a146-6b48c2fdd83a admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:04.982459 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe033d-5425-456f-a146-6b48c2fdd83a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:04.982760 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe033d-5425-456f-a146-6b48c2fdd83a admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:04.983458 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe033d-5425-456f-a146-6b48c2fdd83a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:04.985528 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe033d-5425-456f-a146-6b48c2fdd83a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:05.008424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1081/2159] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:18:04 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:05.013163 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce660211-24c0-4a41-8ab9-ee368527dceb admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:05.023361 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce660211-24c0-4a41-8ab9-ee368527dceb admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:05.027967 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0104c1-7e8b-43d4-b1b6-470104f0f8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:05.028565 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0104c1-7e8b-43d4-b1b6-470104f0f8db admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:05.028847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0104c1-7e8b-43d4-b1b6-470104f0f8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:05.029122 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0104c1-7e8b-43d4-b1b6-470104f0f8db admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:05.029762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0104c1-7e8b-43d4-b1b6-470104f0f8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:05.031422 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0104c1-7e8b-43d4-b1b6-470104f0f8db admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:05.041412 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1079/2160] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:05.045338 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe033d-5425-456f-a146-6b48c2fdd83a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:05.055272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe033d-5425-456f-a146-6b48c2fdd83a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:05.061123 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf8beb-996b-4abe-9cd4-a0aee9ac55be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:05.061768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf8beb-996b-4abe-9cd4-a0aee9ac55be admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:05.062050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf8beb-996b-4abe-9cd4-a0aee9ac55be admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:05.062326 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf8beb-996b-4abe-9cd4-a0aee9ac55be admin admin] PATH_INFO: `/v3/projects/fa546bcf98e242ddbcd1513a949c89c5/users/42340a845c3747808f6fac9f725acc5e/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:05.077801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf8beb-996b-4abe-9cd4-a0aee9ac55be admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa546bcf98e242ddbcd1513a949c89c5, user_id=42340a845c3747808f6fac9f725acc5e, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:05.080488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf8beb-996b-4abe-9cd4-a0aee9ac55be admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:05.093136 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1082/2161] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:05 2022] PUT /identity/v3/projects/fa546bcf98e242ddbcd1513a949c89c5/users/42340a845c3747808f6fac9f725acc5e/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:05.097013 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0104c1-7e8b-43d4-b1b6-470104f0f8db admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:05.106772 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0104c1-7e8b-43d4-b1b6-470104f0f8db admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:05.163253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49eedfa-9c95-4063-bfaf-38ea847c9d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:05.163856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49eedfa-9c95-4063-bfaf-38ea847c9d5c admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:05.164139 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49eedfa-9c95-4063-bfaf-38ea847c9d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:05.164408 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49eedfa-9c95-4063-bfaf-38ea847c9d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:05.165641 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49eedfa-9c95-4063-bfaf-38ea847c9d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:05.167245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49eedfa-9c95-4063-bfaf-38ea847c9d5c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:05.176617 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1080/2162] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:05.180885 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf8beb-996b-4abe-9cd4-a0aee9ac55be admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:05.191077 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf8beb-996b-4abe-9cd4-a0aee9ac55be admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:05.195699 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903ee7d-0c4a-4e72-a0aa-8c75121f19c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:05.196361 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903ee7d-0c4a-4e72-a0aa-8c75121f19c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:05.196716 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903ee7d-0c4a-4e72-a0aa-8c75121f19c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:05.196992 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903ee7d-0c4a-4e72-a0aa-8c75121f19c7 admin admin] PATH_INFO: `/v3/projects/fa546bcf98e242ddbcd1513a949c89c5/users/42340a845c3747808f6fac9f725acc5e/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:05.198921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903ee7d-0c4a-4e72-a0aa-8c75121f19c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa546bcf98e242ddbcd1513a949c89c5, user_id=42340a845c3747808f6fac9f725acc5e, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:05.201575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903ee7d-0c4a-4e72-a0aa-8c75121f19c7 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:05.211527 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1083/2163] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:05 2022] PUT /identity/v3/projects/fa546bcf98e242ddbcd1513a949c89c5/users/42340a845c3747808f6fac9f725acc5e/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:05.223329 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b01d3c-b3dc-4b0a-926a-de77b519edeb None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:05.223624 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b01d3c-b3dc-4b0a-926a-de77b519edeb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:05.223960 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b01d3c-b3dc-4b0a-926a-de77b519edeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:05.252671 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-59b01d3c-b3dc-4b0a-926a-de77b519edeb None None] MFA Rules not processed for user `42340a845c3747808f6fac9f725acc5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:05.253783 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-59b01d3c-b3dc-4b0a-926a-de77b519edeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:05.327765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1081/2164] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:05.339519 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903ee7d-0c4a-4e72-a0aa-8c75121f19c7 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:05.341239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5903ee7d-0c4a-4e72-a0aa-8c75121f19c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:05.355779 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:05.400912 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903ee7d-0c4a-4e72-a0aa-8c75121f19c7 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:05.404562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed08e3-668f-4b78-9c68-301938b6befc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:05.405170 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed08e3-668f-4b78-9c68-301938b6befc service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:05.405452 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed08e3-668f-4b78-9c68-301938b6befc service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:05.405729 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed08e3-668f-4b78-9c68-301938b6befc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:05.415000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed08e3-668f-4b78-9c68-301938b6befc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:05.416488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed08e3-668f-4b78-9c68-301938b6befc service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:05.425861 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1084/2165] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:05.505164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b01d3c-b3dc-4b0a-926a-de77b519edeb None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:05.520472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b01d3c-b3dc-4b0a-926a-de77b519edeb None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:05.524000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344938a6-573f-4452-9663-db3a7303e7c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:05.525307 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344938a6-573f-4452-9663-db3a7303e7c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:05.525581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344938a6-573f-4452-9663-db3a7303e7c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:05.525846 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344938a6-573f-4452-9663-db3a7303e7c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:05.585688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:05.591333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344938a6-573f-4452-9663-db3a7303e7c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:05.592950 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344938a6-573f-4452-9663-db3a7303e7c3 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:05.601745 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1082/2166] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:10.314407 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed08e3-668f-4b78-9c68-301938b6befc service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:10.381282 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed08e3-668f-4b78-9c68-301938b6befc service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:10.386995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc0d6d-2985-4192-b338-2554b33ca091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:10.387628 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc0d6d-2985-4192-b338-2554b33ca091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:10.387911 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc0d6d-2985-4192-b338-2554b33ca091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:10.388185 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc0d6d-2985-4192-b338-2554b33ca091 admin admin] PATH_INFO: `/v3/users/5c7b52cb14ba46e8a534cf038f3bbf3f` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:10.389307 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc0d6d-2985-4192-b338-2554b33ca091 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c7b52cb14ba46e8a534cf038f3bbf3f)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:10.391271 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc0d6d-2985-4192-b338-2554b33ca091 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:10.439801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9bbc0d6d-2985-4192-b338-2554b33ca091 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c7b52cb14ba46e8a534cf038f3bbf3f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:10.447789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9bbc0d6d-2985-4192-b338-2554b33ca091 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c7b52cb14ba46e8a534cf038f3bbf3f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:10.448559 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9bbc0d6d-2985-4192-b338-2554b33ca091 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c7b52cb14ba46e8a534cf038f3bbf3f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:10.457745 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9bbc0d6d-2985-4192-b338-2554b33ca091 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c7b52cb14ba46e8a534cf038f3bbf3f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:10.472160 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9bbc0d6d-2985-4192-b338-2554b33ca091 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c7b52cb14ba46e8a534cf038f3bbf3f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:10.479745 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1085/2167] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:18:10 2022] DELETE /identity/v3/users/5c7b52cb14ba46e8a534cf038f3bbf3f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:10.484480 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f9301-254d-49fc-9aa6-155e9f5421f8 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:10.485009 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f9301-254d-49fc-9aa6-155e9f5421f8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:10.485094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f9301-254d-49fc-9aa6-155e9f5421f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:10.506832 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ec7f9301-254d-49fc-9aa6-155e9f5421f8 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:10.508130 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ec7f9301-254d-49fc-9aa6-155e9f5421f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:10.581530 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1083/2168] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:10.591354 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc0d6d-2985-4192-b338-2554b33ca091 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:10.592783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9bbc0d6d-2985-4192-b338-2554b33ca091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:10.628180 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:10.651004 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc0d6d-2985-4192-b338-2554b33ca091 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:10.654567 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f072c-0dd0-4cbd-b843-2f5b308886f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:10.655212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f072c-0dd0-4cbd-b843-2f5b308886f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:10.655495 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f072c-0dd0-4cbd-b843-2f5b308886f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:10.655770 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f072c-0dd0-4cbd-b843-2f5b308886f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:10.665321 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f072c-0dd0-4cbd-b843-2f5b308886f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:10.666815 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f072c-0dd0-4cbd-b843-2f5b308886f4 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:10.676082 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1086/2169] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:10.953696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f9301-254d-49fc-9aa6-155e9f5421f8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:10.955175 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ec7f9301-254d-49fc-9aa6-155e9f5421f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:10.968230 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f9301-254d-49fc-9aa6-155e9f5421f8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:10.972982 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c869a-0706-442f-90cc-a20d416cfdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:10.973645 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c869a-0706-442f-90cc-a20d416cfdbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:10.973926 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c869a-0706-442f-90cc-a20d416cfdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:10.974201 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c869a-0706-442f-90cc-a20d416cfdbe admin admin] PATH_INFO: `/v3/projects/22430f2c851843e4a64cffb06adc3902` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:10.975403 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c869a-0706-442f-90cc-a20d416cfdbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=22430f2c851843e4a64cffb06adc3902)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:10.977880 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c869a-0706-442f-90cc-a20d416cfdbe admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:11.033276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c93c869a-0706-442f-90cc-a20d416cfdbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22430f2c851843e4a64cffb06adc3902', 'request_id': 'req-c93c869a-0706-442f-90cc-a20d416cfdbe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:11.039334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c93c869a-0706-442f-90cc-a20d416cfdbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22430f2c851843e4a64cffb06adc3902', 'request_id': 'req-c93c869a-0706-442f-90cc-a20d416cfdbe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:11.046389 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c93c869a-0706-442f-90cc-a20d416cfdbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22430f2c851843e4a64cffb06adc3902', 'request_id': 'req-c93c869a-0706-442f-90cc-a20d416cfdbe'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:11.059227 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c93c869a-0706-442f-90cc-a20d416cfdbe admin admin] The token cache is being invalidate because project 22430f2c851843e4a64cffb06adc3902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:18:11.059542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c93c869a-0706-442f-90cc-a20d416cfdbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:11.060281 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c93c869a-0706-442f-90cc-a20d416cfdbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:11.061949 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1084/2170] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:18:10 2022] DELETE /identity/v3/projects/22430f2c851843e4a64cffb06adc3902 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:11.066118 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f072c-0dd0-4cbd-b843-2f5b308886f4 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:11.075704 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f072c-0dd0-4cbd-b843-2f5b308886f4 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:11.130832 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340f664-da4e-43d1-8c13-c5e5fd2c95d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:11.131484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340f664-da4e-43d1-8c13-c5e5fd2c95d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:11.131761 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340f664-da4e-43d1-8c13-c5e5fd2c95d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:11.132032 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340f664-da4e-43d1-8c13-c5e5fd2c95d9 admin admin] PATH_INFO: `/v3/users/42340a845c3747808f6fac9f725acc5e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:11.133147 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3340f664-da4e-43d1-8c13-c5e5fd2c95d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42340a845c3747808f6fac9f725acc5e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:11.135284 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3340f664-da4e-43d1-8c13-c5e5fd2c95d9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:11.200952 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3340f664-da4e-43d1-8c13-c5e5fd2c95d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42340a845c3747808f6fac9f725acc5e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:11.208809 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3340f664-da4e-43d1-8c13-c5e5fd2c95d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42340a845c3747808f6fac9f725acc5e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:11.209554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3340f664-da4e-43d1-8c13-c5e5fd2c95d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42340a845c3747808f6fac9f725acc5e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:11.225387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3340f664-da4e-43d1-8c13-c5e5fd2c95d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42340a845c3747808f6fac9f725acc5e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:11.244215 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3340f664-da4e-43d1-8c13-c5e5fd2c95d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42340a845c3747808f6fac9f725acc5e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:11.251306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1087/2171] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:18:11 2022] DELETE /identity/v3/users/42340a845c3747808f6fac9f725acc5e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:11.544298 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c869a-0706-442f-90cc-a20d416cfdbe admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:11.545944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c93c869a-0706-442f-90cc-a20d416cfdbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:11.560763 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c869a-0706-442f-90cc-a20d416cfdbe admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:11.624248 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179b161-1589-4dd5-80af-87cfe73211b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:11.624891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179b161-1589-4dd5-80af-87cfe73211b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:11.625172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179b161-1589-4dd5-80af-87cfe73211b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:11.625446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179b161-1589-4dd5-80af-87cfe73211b6 admin admin] PATH_INFO: `/v3/projects/fa546bcf98e242ddbcd1513a949c89c5` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:11.626624 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179b161-1589-4dd5-80af-87cfe73211b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa546bcf98e242ddbcd1513a949c89c5)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:11.628536 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179b161-1589-4dd5-80af-87cfe73211b6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:11.651220 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:11.677520 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6179b161-1589-4dd5-80af-87cfe73211b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa546bcf98e242ddbcd1513a949c89c5', 'request_id': 'req-6179b161-1589-4dd5-80af-87cfe73211b6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:11.684145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6179b161-1589-4dd5-80af-87cfe73211b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa546bcf98e242ddbcd1513a949c89c5', 'request_id': 'req-6179b161-1589-4dd5-80af-87cfe73211b6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:11.690716 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6179b161-1589-4dd5-80af-87cfe73211b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa546bcf98e242ddbcd1513a949c89c5', 'request_id': 'req-6179b161-1589-4dd5-80af-87cfe73211b6'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:11.703678 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6179b161-1589-4dd5-80af-87cfe73211b6 admin admin] The token cache is being invalidate because project fa546bcf98e242ddbcd1513a949c89c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:18:11.703973 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6179b161-1589-4dd5-80af-87cfe73211b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:11.704621 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-6179b161-1589-4dd5-80af-87cfe73211b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:11.706369 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1085/2172] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:18:11 2022] DELETE /identity/v3/projects/fa546bcf98e242ddbcd1513a949c89c5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:11.711021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810123f3-2e78-4f1b-ae1c-c427b05b4e2c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:11.711485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810123f3-2e78-4f1b-ae1c-c427b05b4e2c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:11.711637 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810123f3-2e78-4f1b-ae1c-c427b05b4e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:11.732365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-810123f3-2e78-4f1b-ae1c-c427b05b4e2c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:11.733569 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-810123f3-2e78-4f1b-ae1c-c427b05b4e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:11.785058 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:11.799468 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1088/2173] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:11.806830 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa1e19-b34e-4700-9e01-179947c0efd0 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:11.807172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa1e19-b34e-4700-9e01-179947c0efd0 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:11.807456 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa1e19-b34e-4700-9e01-179947c0efd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:11.829243 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-58aa1e19-b34e-4700-9e01-179947c0efd0 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:11.830357 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-58aa1e19-b34e-4700-9e01-179947c0efd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:11.841239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1086/2174] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:11.845272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810123f3-2e78-4f1b-ae1c-c427b05b4e2c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:11.856170 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810123f3-2e78-4f1b-ae1c-c427b05b4e2c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:11.860870 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0380a2-3f85-4070-8d82-0f82df65ef88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:11.861504 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0380a2-3f85-4070-8d82-0f82df65ef88 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:11.861781 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0380a2-3f85-4070-8d82-0f82df65ef88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:11.862050 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0380a2-3f85-4070-8d82-0f82df65ef88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:11.862697 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0380a2-3f85-4070-8d82-0f82df65ef88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:18:11.863038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0380a2-3f85-4070-8d82-0f82df65ef88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:11.864638 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0380a2-3f85-4070-8d82-0f82df65ef88 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:11.877097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1089/2175] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:18:11.881328 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa1e19-b34e-4700-9e01-179947c0efd0 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:11.890835 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa1e19-b34e-4700-9e01-179947c0efd0 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:11.895481 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0611e-2974-48e4-89cd-6481cdd3b6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:11.896054 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0611e-2974-48e4-89cd-6481cdd3b6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:11.896333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0611e-2974-48e4-89cd-6481cdd3b6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:11.896601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0611e-2974-48e4-89cd-6481cdd3b6b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:11.897245 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0611e-2974-48e4-89cd-6481cdd3b6b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:11.899766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0611e-2974-48e4-89cd-6481cdd3b6b7 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:11.930451 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1087/2176] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:18:11 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:11.931614 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0380a2-3f85-4070-8d82-0f82df65ef88 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:11.941682 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0380a2-3f85-4070-8d82-0f82df65ef88 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:11.997939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0f241-2192-4b88-80fa-42dbbe1b82c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:11.998572 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0f241-2192-4b88-80fa-42dbbe1b82c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:11.998852 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0f241-2192-4b88-80fa-42dbbe1b82c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:11.999154 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0f241-2192-4b88-80fa-42dbbe1b82c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:11.999794 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0f241-2192-4b88-80fa-42dbbe1b82c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:12.001759 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0f241-2192-4b88-80fa-42dbbe1b82c7 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:12.023217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1090/2177] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:18:11 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:18:12.028101 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0611e-2974-48e4-89cd-6481cdd3b6b7 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:12.038708 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0611e-2974-48e4-89cd-6481cdd3b6b7 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:12.043374 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02b359-299f-4fb6-a76b-754d3bd248af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:12.043967 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02b359-299f-4fb6-a76b-754d3bd248af admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:12.044242 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02b359-299f-4fb6-a76b-754d3bd248af admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:12.044510 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02b359-299f-4fb6-a76b-754d3bd248af admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:12.045142 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02b359-299f-4fb6-a76b-754d3bd248af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:12.046741 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02b359-299f-4fb6-a76b-754d3bd248af admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:12.056398 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1088/2178] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:12.060336 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0f241-2192-4b88-80fa-42dbbe1b82c7 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:12.070423 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0f241-2192-4b88-80fa-42dbbe1b82c7 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:12.075807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdab8f1-6a44-45e0-9c51-6ac21044a7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:12.076399 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdab8f1-6a44-45e0-9c51-6ac21044a7fb admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:12.076680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdab8f1-6a44-45e0-9c51-6ac21044a7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:12.076955 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdab8f1-6a44-45e0-9c51-6ac21044a7fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:12.077597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdab8f1-6a44-45e0-9c51-6ac21044a7fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:12.079221 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdab8f1-6a44-45e0-9c51-6ac21044a7fb admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:12.090470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1091/2179] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:12.094708 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02b359-299f-4fb6-a76b-754d3bd248af admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:12.104589 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02b359-299f-4fb6-a76b-754d3bd248af admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:12.109117 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8385b-57d3-4939-bbc7-7fae729ae698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:12.109754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8385b-57d3-4939-bbc7-7fae729ae698 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:12.110029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8385b-57d3-4939-bbc7-7fae729ae698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:12.110298 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8385b-57d3-4939-bbc7-7fae729ae698 admin admin] PATH_INFO: `/v3/projects/d8761893c1c54e479b753766f38fe630/users/fe809ee7e6094de788a6faeefea6e59b/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:12.124946 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8385b-57d3-4939-bbc7-7fae729ae698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8761893c1c54e479b753766f38fe630, user_id=fe809ee7e6094de788a6faeefea6e59b, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:12.127604 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8385b-57d3-4939-bbc7-7fae729ae698 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:12.140956 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1089/2180] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:12 2022] PUT /identity/v3/projects/d8761893c1c54e479b753766f38fe630/users/fe809ee7e6094de788a6faeefea6e59b/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:12.151987 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d264319-4ef9-4097-8117-11bcb1878c90 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:12.152271 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d264319-4ef9-4097-8117-11bcb1878c90 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:12.152606 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d264319-4ef9-4097-8117-11bcb1878c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:12.179597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-6d264319-4ef9-4097-8117-11bcb1878c90 None None] MFA Rules not processed for user `fe809ee7e6094de788a6faeefea6e59b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:12.180792 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6d264319-4ef9-4097-8117-11bcb1878c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:12.243602 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1092/2181] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:12.255220 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8385b-57d3-4939-bbc7-7fae729ae698 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:12.256885 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-49d8385b-57d3-4939-bbc7-7fae729ae698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:12.313453 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8385b-57d3-4939-bbc7-7fae729ae698 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:12.317130 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05aa14-b699-4aac-9c83-8eec4de1afcc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8aa29639a3143d2a270e3248d2205d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:12.317745 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05aa14-b699-4aac-9c83-8eec4de1afcc service glance] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:12.318029 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05aa14-b699-4aac-9c83-8eec4de1afcc service glance] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:12.318413 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05aa14-b699-4aac-9c83-8eec4de1afcc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:12.328110 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05aa14-b699-4aac-9c83-8eec4de1afcc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:12.329587 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05aa14-b699-4aac-9c83-8eec4de1afcc service glance] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:12.339079 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1090/2182] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:21.980990 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d264319-4ef9-4097-8117-11bcb1878c90 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:21.982376 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6d264319-4ef9-4097-8117-11bcb1878c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:22.047170 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d264319-4ef9-4097-8117-11bcb1878c90 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:22.050890 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee53c4a-5815-4261-a7f5-7d5949e3c6eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:22.051240 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05aa14-b699-4aac-9c83-8eec4de1afcc service glance] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:22.051682 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee53c4a-5815-4261-a7f5-7d5949e3c6eb service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:22.051817 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee53c4a-5815-4261-a7f5-7d5949e3c6eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:22.052202 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee53c4a-5815-4261-a7f5-7d5949e3c6eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:22.054219 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9ee53c4a-5815-4261-a7f5-7d5949e3c6eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:22.059765 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:22.062188 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05aa14-b699-4aac-9c83-8eec4de1afcc service glance] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:22.066414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989c353-8c93-4412-b3ae-bb23f6609fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:22.067058 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989c353-8c93-4412-b3ae-bb23f6609fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:22.067342 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989c353-8c93-4412-b3ae-bb23f6609fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:22.067616 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989c353-8c93-4412-b3ae-bb23f6609fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:22.069350 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8989c353-8c93-4412-b3ae-bb23f6609fe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:22.118340 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:22.124361 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee53c4a-5815-4261-a7f5-7d5949e3c6eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:22.125881 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee53c4a-5815-4261-a7f5-7d5949e3c6eb service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:22.137003 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1093/2183] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:22.142111 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989c353-8c93-4412-b3ae-bb23f6609fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:22.143681 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989c353-8c93-4412-b3ae-bb23f6609fe5 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:22.154522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1091/2184] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:18:27.225658 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee53c4a-5815-4261-a7f5-7d5949e3c6eb service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:27.227203 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9ee53c4a-5815-4261-a7f5-7d5949e3c6eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:27.287957 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989c353-8c93-4412-b3ae-bb23f6609fe5 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:27.290435 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8989c353-8c93-4412-b3ae-bb23f6609fe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:27.315822 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989c353-8c93-4412-b3ae-bb23f6609fe5 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:27.324001 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bbafc-cb7b-4371-800a-da163b1e7e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:27.324679 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bbafc-cb7b-4371-800a-da163b1e7e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:27.324967 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bbafc-cb7b-4371-800a-da163b1e7e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:27.325247 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bbafc-cb7b-4371-800a-da163b1e7e85 admin admin] PATH_INFO: `/v3/users/1c786544e8fb4e8e8c3a4990a0f36725` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:27.326589 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0bbafc-cb7b-4371-800a-da163b1e7e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c786544e8fb4e8e8c3a4990a0f36725)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:27.328714 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0bbafc-cb7b-4371-800a-da163b1e7e85 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:27.391374 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee53c4a-5815-4261-a7f5-7d5949e3c6eb service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:27.411730 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f92f1e-72e9-448b-b714-2a8285fff7a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cc2655cc32e4d63a7000ef41bcbcfea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:27.412766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f92f1e-72e9-448b-b714-2a8285fff7a2 service placement] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:27.413134 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f92f1e-72e9-448b-b714-2a8285fff7a2 service placement] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:27.413476 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f92f1e-72e9-448b-b714-2a8285fff7a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:27.417251 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-34f92f1e-72e9-448b-b714-2a8285fff7a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:27.419540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4c0bbafc-cb7b-4371-800a-da163b1e7e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c786544e8fb4e8e8c3a4990a0f36725'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:27.428702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4c0bbafc-cb7b-4371-800a-da163b1e7e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c786544e8fb4e8e8c3a4990a0f36725'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:27.429507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4c0bbafc-cb7b-4371-800a-da163b1e7e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c786544e8fb4e8e8c3a4990a0f36725'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:27.435387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f92f1e-72e9-448b-b714-2a8285fff7a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:27.437230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f92f1e-72e9-448b-b714-2a8285fff7a2 service placement] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:27.442003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4c0bbafc-cb7b-4371-800a-da163b1e7e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c786544e8fb4e8e8c3a4990a0f36725'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:27.449734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1094/2185] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:27.460511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4c0bbafc-cb7b-4371-800a-da163b1e7e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c786544e8fb4e8e8c3a4990a0f36725'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:27.474771 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1092/2186] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:18:27 2022] DELETE /identity/v3/users/1c786544e8fb4e8e8c3a4990a0f36725 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:27.590238 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569e237-bf48-4dd0-99e3-5040ca8a0004 None None] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:27.590534 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569e237-bf48-4dd0-99e3-5040ca8a0004 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:27.591266 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569e237-bf48-4dd0-99e3-5040ca8a0004 None None] PATH_INFO: `/` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:27.593109 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1095/2187] 10.222.0.27 () {54 vars in 921 bytes} [Mon Oct 17 03:18:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:18:27.609600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b69453-24de-4988-8070-a3c27a4ebef6 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:27.609990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b69453-24de-4988-8070-a3c27a4ebef6 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:27.610517 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b69453-24de-4988-8070-a3c27a4ebef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:27.640209 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-25b69453-24de-4988-8070-a3c27a4ebef6 None None] MFA Rules not processed for user `4cc2655cc32e4d63a7000ef41bcbcfea`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:27.645803 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569e237-bf48-4dd0-99e3-5040ca8a0004 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:27.648232 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e569e237-bf48-4dd0-99e3-5040ca8a0004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:27.651701 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-25b69453-24de-4988-8070-a3c27a4ebef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:27.749881 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1093/2188] 10.222.0.27 () {58 vars in 1018 bytes} [Mon Oct 17 03:18:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:27.750991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b69453-24de-4988-8070-a3c27a4ebef6 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:27.752585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-25b69453-24de-4988-8070-a3c27a4ebef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:27.791166 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569e237-bf48-4dd0-99e3-5040ca8a0004 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:27.798584 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053f6ad-0ac2-420e-9f97-b27be03f03ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:27.799253 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053f6ad-0ac2-420e-9f97-b27be03f03ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:27.799541 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053f6ad-0ac2-420e-9f97-b27be03f03ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:27.799819 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053f6ad-0ac2-420e-9f97-b27be03f03ee admin admin] PATH_INFO: `/v3/users/ce25db99a7b14d0bbaca41fff80e3da9` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:27.801027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053f6ad-0ac2-420e-9f97-b27be03f03ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce25db99a7b14d0bbaca41fff80e3da9)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:27.803559 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053f6ad-0ac2-420e-9f97-b27be03f03ee admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:27.812587 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:27.831503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b69453-24de-4988-8070-a3c27a4ebef6 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:27.839762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d8ac6-52dd-4513-a641-25fe4bc618b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:27.840500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d8ac6-52dd-4513-a641-25fe4bc618b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:27.840809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d8ac6-52dd-4513-a641-25fe4bc618b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:27.841119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d8ac6-52dd-4513-a641-25fe4bc618b4 admin admin] PATH_INFO: `/v3/projects/618cf0b3507845909bb27c13460d1d6c` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:27.842500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d8ac6-52dd-4513-a641-25fe4bc618b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=618cf0b3507845909bb27c13460d1d6c)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:27.844669 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d8ac6-52dd-4513-a641-25fe4bc618b4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:27.845449 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:27.861874 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b053f6ad-0ac2-420e-9f97-b27be03f03ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce25db99a7b14d0bbaca41fff80e3da9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:27.868801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b053f6ad-0ac2-420e-9f97-b27be03f03ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce25db99a7b14d0bbaca41fff80e3da9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:27.869489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b053f6ad-0ac2-420e-9f97-b27be03f03ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce25db99a7b14d0bbaca41fff80e3da9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:27.877992 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b053f6ad-0ac2-420e-9f97-b27be03f03ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce25db99a7b14d0bbaca41fff80e3da9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:27.891390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b053f6ad-0ac2-420e-9f97-b27be03f03ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce25db99a7b14d0bbaca41fff80e3da9'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:27.896008 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-838d8ac6-52dd-4513-a641-25fe4bc618b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '618cf0b3507845909bb27c13460d1d6c', 'request_id': 'req-838d8ac6-52dd-4513-a641-25fe4bc618b4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:27.900703 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1096/2189] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:18:27 2022] DELETE /identity/v3/users/ce25db99a7b14d0bbaca41fff80e3da9 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:27.900703 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053f6ad-0ac2-420e-9f97-b27be03f03ee admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:27.900703 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b053f6ad-0ac2-420e-9f97-b27be03f03ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:27.908079 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-838d8ac6-52dd-4513-a641-25fe4bc618b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '618cf0b3507845909bb27c13460d1d6c', 'request_id': 'req-838d8ac6-52dd-4513-a641-25fe4bc618b4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:27.915249 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-838d8ac6-52dd-4513-a641-25fe4bc618b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '618cf0b3507845909bb27c13460d1d6c', 'request_id': 'req-838d8ac6-52dd-4513-a641-25fe4bc618b4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:27.931844 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-838d8ac6-52dd-4513-a641-25fe4bc618b4 admin admin] The token cache is being invalidate because project 618cf0b3507845909bb27c13460d1d6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:18:27.933059 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-838d8ac6-52dd-4513-a641-25fe4bc618b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:27.934176 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-838d8ac6-52dd-4513-a641-25fe4bc618b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:27.936857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1094/2190] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:18:27 2022] DELETE /identity/v3/projects/618cf0b3507845909bb27c13460d1d6c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:27.942819 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bc421-9c79-47e9-a16c-81564016430c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:27.943142 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bc421-9c79-47e9-a16c-81564016430c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:27.943478 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bc421-9c79-47e9-a16c-81564016430c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:27.965882 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-856bc421-9c79-47e9-a16c-81564016430c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:27.967125 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-856bc421-9c79-47e9-a16c-81564016430c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:27.980320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053f6ad-0ac2-420e-9f97-b27be03f03ee admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:27.986088 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12217b92-5bb4-44b6-830f-0bc7bd0aea70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cc2655cc32e4d63a7000ef41bcbcfea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:27.987222 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12217b92-5bb4-44b6-830f-0bc7bd0aea70 service placement] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:27.987628 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12217b92-5bb4-44b6-830f-0bc7bd0aea70 service placement] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:27.987971 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12217b92-5bb4-44b6-830f-0bc7bd0aea70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:27.989963 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-12217b92-5bb4-44b6-830f-0bc7bd0aea70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:28.005950 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12217b92-5bb4-44b6-830f-0bc7bd0aea70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.007701 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12217b92-5bb4-44b6-830f-0bc7bd0aea70 service placement] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.019970 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1097/2191] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:28.038542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1095/2192] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:18:28.039967 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c113933-d90b-401d-960a-c76fa2b19e3d None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.040258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c113933-d90b-401d-960a-c76fa2b19e3d None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.040543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c113933-d90b-401d-960a-c76fa2b19e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.062542 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3c113933-d90b-401d-960a-c76fa2b19e3d None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:28.063800 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3c113933-d90b-401d-960a-c76fa2b19e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:28.071750 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1098/2193] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:28.076547 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bc421-9c79-47e9-a16c-81564016430c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.087041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bc421-9c79-47e9-a16c-81564016430c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.092319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e8474-18de-4332-be47-8817a9df4f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.093032 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e8474-18de-4332-be47-8817a9df4f29 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.093329 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e8474-18de-4332-be47-8817a9df4f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.093622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e8474-18de-4332-be47-8817a9df4f29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.094360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e8474-18de-4332-be47-8817a9df4f29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:18:28.094705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e8474-18de-4332-be47-8817a9df4f29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.096584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e8474-18de-4332-be47-8817a9df4f29 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.104006 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c113933-d90b-401d-960a-c76fa2b19e3d None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.105682 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3c113933-d90b-401d-960a-c76fa2b19e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:28.111527 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1096/2194] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:18:28.117588 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e8474-18de-4332-be47-8817a9df4f29 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.123511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c113933-d90b-401d-960a-c76fa2b19e3d None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.128688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e8474-18de-4332-be47-8817a9df4f29 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.128996 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2671fb-ed66-4977-9fa6-0389cce7eb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.129714 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2671fb-ed66-4977-9fa6-0389cce7eb53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.130117 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2671fb-ed66-4977-9fa6-0389cce7eb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.131007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2671fb-ed66-4977-9fa6-0389cce7eb53 admin admin] PATH_INFO: `/v3/projects/e5bad80dc0d74119b2a9cd5c1935e8d5` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.131742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2671fb-ed66-4977-9fa6-0389cce7eb53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5bad80dc0d74119b2a9cd5c1935e8d5)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.134005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2671fb-ed66-4977-9fa6-0389cce7eb53 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.139355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be517b54-a5f9-40ba-8f9b-de997c2c2aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.140000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be517b54-a5f9-40ba-8f9b-de997c2c2aec admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.140283 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be517b54-a5f9-40ba-8f9b-de997c2c2aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.140557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be517b54-a5f9-40ba-8f9b-de997c2c2aec admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.141202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be517b54-a5f9-40ba-8f9b-de997c2c2aec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.143161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be517b54-a5f9-40ba-8f9b-de997c2c2aec admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.174781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1097/2195] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:18:28 2022] POST /identity/v3/projects => generated 401 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:28.175862 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be517b54-a5f9-40ba-8f9b-de997c2c2aec admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.186511 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0f2671fb-ed66-4977-9fa6-0389cce7eb53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5bad80dc0d74119b2a9cd5c1935e8d5', 'request_id': 'req-0f2671fb-ed66-4977-9fa6-0389cce7eb53'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:28.192293 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0f2671fb-ed66-4977-9fa6-0389cce7eb53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5bad80dc0d74119b2a9cd5c1935e8d5', 'request_id': 'req-0f2671fb-ed66-4977-9fa6-0389cce7eb53'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:28.200067 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0f2671fb-ed66-4977-9fa6-0389cce7eb53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5bad80dc0d74119b2a9cd5c1935e8d5', 'request_id': 'req-0f2671fb-ed66-4977-9fa6-0389cce7eb53'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:28.213787 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0f2671fb-ed66-4977-9fa6-0389cce7eb53 admin admin] The token cache is being invalidate because project e5bad80dc0d74119b2a9cd5c1935e8d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:18:28.214049 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0f2671fb-ed66-4977-9fa6-0389cce7eb53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:28.214706 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-0f2671fb-ed66-4977-9fa6-0389cce7eb53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:28.216397 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1099/2196] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:18:28 2022] DELETE /identity/v3/projects/e5bad80dc0d74119b2a9cd5c1935e8d5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:18:28.220997 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd816777-0519-486d-abe0-b3c9c5eff0de None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.221280 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd816777-0519-486d-abe0-b3c9c5eff0de None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.221554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd816777-0519-486d-abe0-b3c9c5eff0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.227194 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:28.242196 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-dd816777-0519-486d-abe0-b3c9c5eff0de None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:28.243433 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-dd816777-0519-486d-abe0-b3c9c5eff0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:28.248266 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be517b54-a5f9-40ba-8f9b-de997c2c2aec admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.252113 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1100/2197] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:18:28.252985 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65434491-41dd-42c1-9884-61e1f9237853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.253586 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65434491-41dd-42c1-9884-61e1f9237853 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.253864 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65434491-41dd-42c1-9884-61e1f9237853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.254136 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65434491-41dd-42c1-9884-61e1f9237853 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.254764 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65434491-41dd-42c1-9884-61e1f9237853 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.256748 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65434491-41dd-42c1-9884-61e1f9237853 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.262083 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e97222-9177-4964-b3e4-5efd8c641ede None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.262374 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e97222-9177-4964-b3e4-5efd8c641ede None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.262655 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e97222-9177-4964-b3e4-5efd8c641ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.281910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1098/2198] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:18:28 2022] POST /identity/v3/users => generated 301 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:18:28.283038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-98e97222-9177-4964-b3e4-5efd8c641ede None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:28.284340 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-98e97222-9177-4964-b3e4-5efd8c641ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:28.285026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65434491-41dd-42c1-9884-61e1f9237853 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.287005 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-65434491-41dd-42c1-9884-61e1f9237853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:28.293027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1101/2199] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:28.297038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e97222-9177-4964-b3e4-5efd8c641ede None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.301555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:28.302893 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65434491-41dd-42c1-9884-61e1f9237853 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.308687 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e97222-9177-4964-b3e4-5efd8c641ede None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.308846 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441495d-02e3-43d4-acd0-b1565accb409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.309393 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441495d-02e3-43d4-acd0-b1565accb409 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.309669 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441495d-02e3-43d4-acd0-b1565accb409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.309943 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441495d-02e3-43d4-acd0-b1565accb409 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.310651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441495d-02e3-43d4-acd0-b1565accb409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.312589 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441495d-02e3-43d4-acd0-b1565accb409 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.314362 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e1263-5c3e-4049-b862-b475ec06d6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.314985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e1263-5c3e-4049-b862-b475ec06d6df admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.315264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e1263-5c3e-4049-b862-b475ec06d6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.315535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e1263-5c3e-4049-b862-b475ec06d6df admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.316174 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e1263-5c3e-4049-b862-b475ec06d6df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:18:28.316476 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e1263-5c3e-4049-b862-b475ec06d6df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.318060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e1263-5c3e-4049-b862-b475ec06d6df admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.322791 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1099/2200] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:18:28.327788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441495d-02e3-43d4-acd0-b1565accb409 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.330622 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1102/2201] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:18:28.334487 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e1263-5c3e-4049-b862-b475ec06d6df admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.338539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441495d-02e3-43d4-acd0-b1565accb409 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.343823 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf679a-5337-4ab8-9209-69ac445265d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.344415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf679a-5337-4ab8-9209-69ac445265d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.344686 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e1263-5c3e-4049-b862-b475ec06d6df admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.344806 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf679a-5337-4ab8-9209-69ac445265d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.344954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf679a-5337-4ab8-9209-69ac445265d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.345584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf679a-5337-4ab8-9209-69ac445265d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.347194 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf679a-5337-4ab8-9209-69ac445265d8 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.349472 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef4b9f-11b6-4c59-a2b8-b068c0c9d9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.350055 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef4b9f-11b6-4c59-a2b8-b068c0c9d9e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.350337 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef4b9f-11b6-4c59-a2b8-b068c0c9d9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.350610 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef4b9f-11b6-4c59-a2b8-b068c0c9d9e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.351277 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef4b9f-11b6-4c59-a2b8-b068c0c9d9e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.353227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef4b9f-11b6-4c59-a2b8-b068c0c9d9e1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.357726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1100/2202] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:18:28.361780 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf679a-5337-4ab8-9209-69ac445265d8 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.371561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf679a-5337-4ab8-9209-69ac445265d8 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.376499 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb56757-31bc-4123-bedf-9c279997e59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.377164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb56757-31bc-4123-bedf-9c279997e59e admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.377437 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb56757-31bc-4123-bedf-9c279997e59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.377693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb56757-31bc-4123-bedf-9c279997e59e admin admin] PATH_INFO: `/v3/projects/4f9bd622b30c4e3cb47ff332452c63e1/users/8975087c65094417a5b7068c89128ed4/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.388101 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1103/2203] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:18:28 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:18:28.389151 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef4b9f-11b6-4c59-a2b8-b068c0c9d9e1 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.395791 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb56757-31bc-4123-bedf-9c279997e59e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f9bd622b30c4e3cb47ff332452c63e1, user_id=8975087c65094417a5b7068c89128ed4, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.398473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb56757-31bc-4123-bedf-9c279997e59e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.411833 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1101/2204] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:28 2022] PUT /identity/v3/projects/4f9bd622b30c4e3cb47ff332452c63e1/users/8975087c65094417a5b7068c89128ed4/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:18:28.422622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9c5b1-7754-48f7-95d9-39c31bf4e601 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.422942 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9c5b1-7754-48f7-95d9-39c31bf4e601 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.423223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9c5b1-7754-48f7-95d9-39c31bf4e601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.453145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2eb9c5b1-7754-48f7-95d9-39c31bf4e601 None None] MFA Rules not processed for user `8975087c65094417a5b7068c89128ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:28.454350 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2eb9c5b1-7754-48f7-95d9-39c31bf4e601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:28.455795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef4b9f-11b6-4c59-a2b8-b068c0c9d9e1 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.512960 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc77494-5a84-4db2-bfb9-cd610192cb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.513583 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc77494-5a84-4db2-bfb9-cd610192cb2b admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.513862 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc77494-5a84-4db2-bfb9-cd610192cb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.514140 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc77494-5a84-4db2-bfb9-cd610192cb2b admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.514782 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc77494-5a84-4db2-bfb9-cd610192cb2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.516757 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc77494-5a84-4db2-bfb9-cd610192cb2b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.518710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1102/2205] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:28 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:18:28.523174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9c5b1-7754-48f7-95d9-39c31bf4e601 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.534251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9c5b1-7754-48f7-95d9-39c31bf4e601 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.542200 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1104/2206] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:18:28 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:28.542375 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf478a83-ed7f-4ca1-ba02-1c5134b30433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.542375 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf478a83-ed7f-4ca1-ba02-1c5134b30433 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.542375 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf478a83-ed7f-4ca1-ba02-1c5134b30433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.542375 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf478a83-ed7f-4ca1-ba02-1c5134b30433 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.542773 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf478a83-ed7f-4ca1-ba02-1c5134b30433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.543123 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc77494-5a84-4db2-bfb9-cd610192cb2b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.543234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf478a83-ed7f-4ca1-ba02-1c5134b30433 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.552570 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc77494-5a84-4db2-bfb9-cd610192cb2b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.557222 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb1981-5949-404a-8165-f894e05603b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.558312 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb1981-5949-404a-8165-f894e05603b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.558596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb1981-5949-404a-8165-f894e05603b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.558885 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb1981-5949-404a-8165-f894e05603b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.559547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb1981-5949-404a-8165-f894e05603b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.561226 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acb1981-5949-404a-8165-f894e05603b2 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.571547 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1103/2207] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:18:28 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:18:28.571723 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1105/2208] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:18:28.575858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf478a83-ed7f-4ca1-ba02-1c5134b30433 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.575985 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb1981-5949-404a-8165-f894e05603b2 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.585573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acb1981-5949-404a-8165-f894e05603b2 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.632988 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:28.636416 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:28.639608 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf478a83-ed7f-4ca1-ba02-1c5134b30433 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.645133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d34f1c-9546-473e-8e86-3e5eeaba74f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.645753 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d34f1c-9546-473e-8e86-3e5eeaba74f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.646035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d34f1c-9546-473e-8e86-3e5eeaba74f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.646311 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d34f1c-9546-473e-8e86-3e5eeaba74f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.646995 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d34f1c-9546-473e-8e86-3e5eeaba74f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.647880 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd2203-9a19-4a88-85ae-54d5010d2788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.648484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd2203-9a19-4a88-85ae-54d5010d2788 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.648764 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd2203-9a19-4a88-85ae-54d5010d2788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.648879 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d34f1c-9546-473e-8e86-3e5eeaba74f4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.649043 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd2203-9a19-4a88-85ae-54d5010d2788 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.649665 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd2203-9a19-4a88-85ae-54d5010d2788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.652080 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd2203-9a19-4a88-85ae-54d5010d2788 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.659113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1104/2209] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:28.664122 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d34f1c-9546-473e-8e86-3e5eeaba74f4 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.675182 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d34f1c-9546-473e-8e86-3e5eeaba74f4 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.679515 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1106/2210] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:18:28 2022] POST /identity/v3/users => generated 307 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:18:28.680048 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d6f4b-a3e2-4443-9c76-f35266a5d626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.680696 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d6f4b-a3e2-4443-9c76-f35266a5d626 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.680814 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd2203-9a19-4a88-85ae-54d5010d2788 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.680981 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d6f4b-a3e2-4443-9c76-f35266a5d626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.681249 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d6f4b-a3e2-4443-9c76-f35266a5d626 admin admin] PATH_INFO: `/v3/projects/3836eae073a146a796fd7da052ed45cd/users/16a5c17cd80c497d8453bcadceb091c4/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.691052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd2203-9a19-4a88-85ae-54d5010d2788 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.696157 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9fde8e-360c-4975-9b68-18f225167066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.696760 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fde8e-360c-4975-9b68-18f225167066 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.696948 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d6f4b-a3e2-4443-9c76-f35266a5d626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3836eae073a146a796fd7da052ed45cd, user_id=16a5c17cd80c497d8453bcadceb091c4, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.697086 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fde8e-360c-4975-9b68-18f225167066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.697308 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9fde8e-360c-4975-9b68-18f225167066 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.698166 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9fde8e-360c-4975-9b68-18f225167066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.699758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d6f4b-a3e2-4443-9c76-f35266a5d626 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.699867 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9fde8e-360c-4975-9b68-18f225167066 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.711315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1107/2211] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:18:28.712261 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1105/2212] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:28 2022] PUT /identity/v3/projects/3836eae073a146a796fd7da052ed45cd/users/16a5c17cd80c497d8453bcadceb091c4/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:18:28.715887 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9fde8e-360c-4975-9b68-18f225167066 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.717004 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f01f6-22b1-4e69-9e2c-6fa72f1d0a43 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.717291 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f01f6-22b1-4e69-9e2c-6fa72f1d0a43 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.717569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f01f6-22b1-4e69-9e2c-6fa72f1d0a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.725591 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9fde8e-360c-4975-9b68-18f225167066 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.736743 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7b3f01f6-22b1-4e69-9e2c-6fa72f1d0a43 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:28.737931 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7b3f01f6-22b1-4e69-9e2c-6fa72f1d0a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:28.789125 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b636f8-86d3-4f3e-be9f-549392751644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.789856 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b636f8-86d3-4f3e-be9f-549392751644 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.790204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b636f8-86d3-4f3e-be9f-549392751644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.790546 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b636f8-86d3-4f3e-be9f-549392751644 admin admin] PATH_INFO: `/v3/projects/9c18d84f2a3647e48d8b4d10ec3f36d2/users/ccf5a72fe97c401f8e8349a027488f75/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.806902 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1106/2213] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:28.817447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f01f6-22b1-4e69-9e2c-6fa72f1d0a43 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.817863 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b636f8-86d3-4f3e-be9f-549392751644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c18d84f2a3647e48d8b4d10ec3f36d2, user_id=ccf5a72fe97c401f8e8349a027488f75, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.819103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7b3f01f6-22b1-4e69-9e2c-6fa72f1d0a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:28.820676 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b636f8-86d3-4f3e-be9f-549392751644 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.836951 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1108/2214] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:28 2022] PUT /identity/v3/projects/9c18d84f2a3647e48d8b4d10ec3f36d2/users/ccf5a72fe97c401f8e8349a027488f75/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:18:28.841504 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b636f8-86d3-4f3e-be9f-549392751644 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.884452 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f01f6-22b1-4e69-9e2c-6fa72f1d0a43 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.913425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b636f8-86d3-4f3e-be9f-549392751644 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.918566 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd0e78-eea5-4fc9-96da-a168a6c28714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.919222 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd0e78-eea5-4fc9-96da-a168a6c28714 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.919523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd0e78-eea5-4fc9-96da-a168a6c28714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.919801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd0e78-eea5-4fc9-96da-a168a6c28714 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.920446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd0e78-eea5-4fc9-96da-a168a6c28714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.922049 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd0e78-eea5-4fc9-96da-a168a6c28714 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.943768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1109/2215] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:28.943925 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914e024-f04d-4b9a-a2d4-135cc3955e9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.943925 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914e024-f04d-4b9a-a2d4-135cc3955e9c service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.943925 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914e024-f04d-4b9a-a2d4-135cc3955e9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.943925 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914e024-f04d-4b9a-a2d4-135cc3955e9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.944431 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd0e78-eea5-4fc9-96da-a168a6c28714 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:28.952742 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914e024-f04d-4b9a-a2d4-135cc3955e9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.955057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914e024-f04d-4b9a-a2d4-135cc3955e9c service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.956771 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd0e78-eea5-4fc9-96da-a168a6c28714 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:28.963195 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f60693-a325-42cb-8669-74a16d5e0233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:28.963877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f60693-a325-42cb-8669-74a16d5e0233 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.964163 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f60693-a325-42cb-8669-74a16d5e0233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.964442 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f60693-a325-42cb-8669-74a16d5e0233 admin admin] PATH_INFO: `/v3/projects/9c18d84f2a3647e48d8b4d10ec3f36d2/users/ccf5a72fe97c401f8e8349a027488f75/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:28.966384 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f60693-a325-42cb-8669-74a16d5e0233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c18d84f2a3647e48d8b4d10ec3f36d2, user_id=ccf5a72fe97c401f8e8349a027488f75, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:28.966848 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1107/2216] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:18:28.969076 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f60693-a325-42cb-8669-74a16d5e0233 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:28.982460 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1110/2217] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:28 2022] PUT /identity/v3/projects/9c18d84f2a3647e48d8b4d10ec3f36d2/users/ccf5a72fe97c401f8e8349a027488f75/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:18:28.994508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf37e7a-71a7-4d5a-bca9-2ee9a36299d4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:28.995701 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf37e7a-71a7-4d5a-bca9-2ee9a36299d4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:28.995701 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf37e7a-71a7-4d5a-bca9-2ee9a36299d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:29.036425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-fbf37e7a-71a7-4d5a-bca9-2ee9a36299d4 None None] MFA Rules not processed for user `ccf5a72fe97c401f8e8349a027488f75`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:29.037854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fbf37e7a-71a7-4d5a-bca9-2ee9a36299d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:29.053098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:29.126920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1108/2218] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:28 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:29.138046 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f60693-a325-42cb-8669-74a16d5e0233 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:29.140949 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b1f60693-a325-42cb-8669-74a16d5e0233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:29.225150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:29.248585 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f60693-a325-42cb-8669-74a16d5e0233 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:29.252537 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38c3f0-dd19-4bc6-9ee4-95a7c517208e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:29.253685 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38c3f0-dd19-4bc6-9ee4-95a7c517208e service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:29.253975 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38c3f0-dd19-4bc6-9ee4-95a7c517208e service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:29.254277 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38c3f0-dd19-4bc6-9ee4-95a7c517208e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:29.264409 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38c3f0-dd19-4bc6-9ee4-95a7c517208e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:29.265901 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e38c3f0-dd19-4bc6-9ee4-95a7c517208e service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:29.281286 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1111/2219] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:29 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:29.368814 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf37e7a-71a7-4d5a-bca9-2ee9a36299d4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:29.380460 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf37e7a-71a7-4d5a-bca9-2ee9a36299d4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:29.384476 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd3373-894b-4cb0-a47e-90746ea99d0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:29.385115 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd3373-894b-4cb0-a47e-90746ea99d0f service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:29.385416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd3373-894b-4cb0-a47e-90746ea99d0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:29.385657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd3373-894b-4cb0-a47e-90746ea99d0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:29.458093 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd3373-894b-4cb0-a47e-90746ea99d0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:29.459957 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd3373-894b-4cb0-a47e-90746ea99d0f service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:29.470007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1109/2220] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:29 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:29.732930 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21562307-e030-499f-97bd-9a52dfa01f73 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:29.733218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21562307-e030-499f-97bd-9a52dfa01f73 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:29.733496 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21562307-e030-499f-97bd-9a52dfa01f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:29.782087 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-21562307-e030-499f-97bd-9a52dfa01f73 None None] MFA Rules not processed for user `16a5c17cd80c497d8453bcadceb091c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:29.783523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-21562307-e030-499f-97bd-9a52dfa01f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:29.868003 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1112/2221] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:29.873086 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd3373-894b-4cb0-a47e-90746ea99d0f service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:29.935800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd3373-894b-4cb0-a47e-90746ea99d0f service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:29.940251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9ec3f-d9b5-41e7-b702-59e8f09ed1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:29.940852 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9ec3f-d9b5-41e7-b702-59e8f09ed1ce admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:29.941129 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9ec3f-d9b5-41e7-b702-59e8f09ed1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:29.941400 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9ec3f-d9b5-41e7-b702-59e8f09ed1ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:29.942035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9ec3f-d9b5-41e7-b702-59e8f09ed1ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:29.943960 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9ec3f-d9b5-41e7-b702-59e8f09ed1ce admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:29.969391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1110/2222] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:18:29 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:29.973665 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21562307-e030-499f-97bd-9a52dfa01f73 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:29.984861 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21562307-e030-499f-97bd-9a52dfa01f73 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:30.045121 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f5f33-fe40-426c-abbf-49dfc86f5023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:30.045725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f5f33-fe40-426c-abbf-49dfc86f5023 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:30.046003 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f5f33-fe40-426c-abbf-49dfc86f5023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:30.046273 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f5f33-fe40-426c-abbf-49dfc86f5023 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:30.046929 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f5f33-fe40-426c-abbf-49dfc86f5023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:30.048838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f5f33-fe40-426c-abbf-49dfc86f5023 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:30.069958 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1113/2223] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:18:29 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:30.074051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9ec3f-d9b5-41e7-b702-59e8f09ed1ce admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:30.083964 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9ec3f-d9b5-41e7-b702-59e8f09ed1ce admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:30.088514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fcf77-44a0-494c-81ff-10f2939bd991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:30.089127 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fcf77-44a0-494c-81ff-10f2939bd991 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:30.089494 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fcf77-44a0-494c-81ff-10f2939bd991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:30.089811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fcf77-44a0-494c-81ff-10f2939bd991 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:30.090464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546fcf77-44a0-494c-81ff-10f2939bd991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:30.092352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546fcf77-44a0-494c-81ff-10f2939bd991 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:30.102985 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1111/2224] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:30.107683 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f5f33-fe40-426c-abbf-49dfc86f5023 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:30.117491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f5f33-fe40-426c-abbf-49dfc86f5023 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:30.122841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1449c-128a-470c-aa04-fb18c2c693fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:30.123506 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1449c-128a-470c-aa04-fb18c2c693fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:30.123784 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1449c-128a-470c-aa04-fb18c2c693fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:30.124056 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1449c-128a-470c-aa04-fb18c2c693fa admin admin] PATH_INFO: `/v3/projects/674e3602e0b3411cb7db1130b93041cf/users/9655f3864c2f42b2b113bf372e92e47c/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:30.138926 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1449c-128a-470c-aa04-fb18c2c693fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=674e3602e0b3411cb7db1130b93041cf, user_id=9655f3864c2f42b2b113bf372e92e47c, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:30.141634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1449c-128a-470c-aa04-fb18c2c693fa admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:30.154717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1114/2225] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:30 2022] PUT /identity/v3/projects/674e3602e0b3411cb7db1130b93041cf/users/9655f3864c2f42b2b113bf372e92e47c/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:30.158594 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fcf77-44a0-494c-81ff-10f2939bd991 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:30.169234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fcf77-44a0-494c-81ff-10f2939bd991 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:30.189233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:30.227549 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162b167-f2ee-43de-ac96-dc873f4188e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:30.228155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162b167-f2ee-43de-ac96-dc873f4188e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:30.228431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162b167-f2ee-43de-ac96-dc873f4188e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:30.228700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162b167-f2ee-43de-ac96-dc873f4188e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:30.229886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162b167-f2ee-43de-ac96-dc873f4188e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:30.231500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162b167-f2ee-43de-ac96-dc873f4188e7 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:30.241131 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1112/2226] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:30.245956 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1449c-128a-470c-aa04-fb18c2c693fa admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:30.256451 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1449c-128a-470c-aa04-fb18c2c693fa admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:30.261399 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c22c13-30d6-4a73-8d5e-7438e06fd6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:30.262051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c22c13-30d6-4a73-8d5e-7438e06fd6af admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:30.262331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c22c13-30d6-4a73-8d5e-7438e06fd6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:30.262605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c22c13-30d6-4a73-8d5e-7438e06fd6af admin admin] PATH_INFO: `/v3/projects/674e3602e0b3411cb7db1130b93041cf/users/9655f3864c2f42b2b113bf372e92e47c/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:30.264623 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c22c13-30d6-4a73-8d5e-7438e06fd6af admin admin] RBAC: Authorizing `identity:create_grant(project_id=674e3602e0b3411cb7db1130b93041cf, user_id=9655f3864c2f42b2b113bf372e92e47c, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:30.267550 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c22c13-30d6-4a73-8d5e-7438e06fd6af admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:30.269094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162b167-f2ee-43de-ac96-dc873f4188e7 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:30.270790 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d162b167-f2ee-43de-ac96-dc873f4188e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:30.283536 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1115/2227] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:30 2022] PUT /identity/v3/projects/674e3602e0b3411cb7db1130b93041cf/users/9655f3864c2f42b2b113bf372e92e47c/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:18:30.287323 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162b167-f2ee-43de-ac96-dc873f4188e7 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:30.344794 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b073658-e526-446a-bf4d-15065dad8f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:30.345436 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b073658-e526-446a-bf4d-15065dad8f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:30.345712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b073658-e526-446a-bf4d-15065dad8f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:30.345982 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b073658-e526-446a-bf4d-15065dad8f80 admin admin] PATH_INFO: `/v3/users/fe809ee7e6094de788a6faeefea6e59b` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:30.347157 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b073658-e526-446a-bf4d-15065dad8f80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe809ee7e6094de788a6faeefea6e59b)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:30.349102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b073658-e526-446a-bf4d-15065dad8f80 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:30.396424 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7b073658-e526-446a-bf4d-15065dad8f80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe809ee7e6094de788a6faeefea6e59b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:30.403977 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7b073658-e526-446a-bf4d-15065dad8f80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe809ee7e6094de788a6faeefea6e59b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:30.404686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7b073658-e526-446a-bf4d-15065dad8f80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe809ee7e6094de788a6faeefea6e59b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:30.412710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7b073658-e526-446a-bf4d-15065dad8f80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe809ee7e6094de788a6faeefea6e59b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:30.425847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7b073658-e526-446a-bf4d-15065dad8f80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe809ee7e6094de788a6faeefea6e59b'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:30.432222 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1113/2228] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:18:30 2022] DELETE /identity/v3/users/fe809ee7e6094de788a6faeefea6e59b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:30.436767 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6430fb-1f85-4c81-90fa-a6a2cc95f993 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:30.437448 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6430fb-1f85-4c81-90fa-a6a2cc95f993 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:30.437975 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6430fb-1f85-4c81-90fa-a6a2cc95f993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:30.460597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ec6430fb-1f85-4c81-90fa-a6a2cc95f993 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:30.462104 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ec6430fb-1f85-4c81-90fa-a6a2cc95f993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:30.482046 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:30.532113 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1116/2229] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:30.538292 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b073658-e526-446a-bf4d-15065dad8f80 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:30.539753 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7b073658-e526-446a-bf4d-15065dad8f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:30.589394 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:30.607390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b073658-e526-446a-bf4d-15065dad8f80 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:30.611497 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ccc35-60a3-4a15-93ab-3afb171ece16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:30.612145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ccc35-60a3-4a15-93ab-3afb171ece16 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:30.612425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ccc35-60a3-4a15-93ab-3afb171ece16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:30.612697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ccc35-60a3-4a15-93ab-3afb171ece16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:30.622385 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ccc35-60a3-4a15-93ab-3afb171ece16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:30.623867 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ccc35-60a3-4a15-93ab-3afb171ece16 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:30.634159 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1114/2230] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:31.124391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6430fb-1f85-4c81-90fa-a6a2cc95f993 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:31.126149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ec6430fb-1f85-4c81-90fa-a6a2cc95f993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:31.152996 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6430fb-1f85-4c81-90fa-a6a2cc95f993 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:31.165290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715e23a-42b8-4fd8-9929-efe4b1ff1437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:31.167944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc0acd-6e2e-44fc-b530-4a37a046e216 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:31.168097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715e23a-42b8-4fd8-9929-efe4b1ff1437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:31.168289 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715e23a-42b8-4fd8-9929-efe4b1ff1437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:31.168765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc0acd-6e2e-44fc-b530-4a37a046e216 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:31.169233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc0acd-6e2e-44fc-b530-4a37a046e216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:31.169415 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715e23a-42b8-4fd8-9929-efe4b1ff1437 admin admin] PATH_INFO: `/v3/projects/d8761893c1c54e479b753766f38fe630` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:31.171486 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715e23a-42b8-4fd8-9929-efe4b1ff1437 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8761893c1c54e479b753766f38fe630)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:31.175116 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715e23a-42b8-4fd8-9929-efe4b1ff1437 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:31.249616 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2ffc0acd-6e2e-44fc-b530-4a37a046e216 None None] MFA Rules not processed for user `9655f3864c2f42b2b113bf372e92e47c`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:31.252094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2ffc0acd-6e2e-44fc-b530-4a37a046e216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:31.295230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b715e23a-42b8-4fd8-9929-efe4b1ff1437 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8761893c1c54e479b753766f38fe630', 'request_id': 'req-b715e23a-42b8-4fd8-9929-efe4b1ff1437'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:31.295230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b715e23a-42b8-4fd8-9929-efe4b1ff1437 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8761893c1c54e479b753766f38fe630', 'request_id': 'req-b715e23a-42b8-4fd8-9929-efe4b1ff1437'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:31.302302 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b715e23a-42b8-4fd8-9929-efe4b1ff1437 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8761893c1c54e479b753766f38fe630', 'request_id': 'req-b715e23a-42b8-4fd8-9929-efe4b1ff1437'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:31.326967 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b715e23a-42b8-4fd8-9929-efe4b1ff1437 admin admin] The token cache is being invalidate because project d8761893c1c54e479b753766f38fe630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:18:31.327271 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b715e23a-42b8-4fd8-9929-efe4b1ff1437 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:31.329091 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-b715e23a-42b8-4fd8-9929-efe4b1ff1437 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:31.332263 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1117/2231] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:18:31 2022] DELETE /identity/v3/projects/d8761893c1c54e479b753766f38fe630 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:31.337864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92a943-90c5-4be8-87cd-d6bd7dc734b6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:31.338151 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92a943-90c5-4be8-87cd-d6bd7dc734b6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:31.338459 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92a943-90c5-4be8-87cd-d6bd7dc734b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:31.362885 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-be92a943-90c5-4be8-87cd-d6bd7dc734b6 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:31.364118 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-be92a943-90c5-4be8-87cd-d6bd7dc734b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:31.378374 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1115/2232] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:31.397587 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc0acd-6e2e-44fc-b530-4a37a046e216 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:31.400020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2ffc0acd-6e2e-44fc-b530-4a37a046e216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:31.462006 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1118/2233] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:18:31.463629 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbd382-5b09-4fd4-a551-92da84f4bc8a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:31.465501 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbd382-5b09-4fd4-a551-92da84f4bc8a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:31.465629 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbd382-5b09-4fd4-a551-92da84f4bc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:31.471688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc0acd-6e2e-44fc-b530-4a37a046e216 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:31.477016 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3851381-c9b4-4d66-aec9-e09d7456c72f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:31.477617 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3851381-c9b4-4d66-aec9-e09d7456c72f service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:31.477804 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3851381-c9b4-4d66-aec9-e09d7456c72f service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:31.478198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3851381-c9b4-4d66-aec9-e09d7456c72f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:31.480624 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c3851381-c9b4-4d66-aec9-e09d7456c72f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:31.487634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-26bbd382-5b09-4fd4-a551-92da84f4bc8a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:31.489589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-26bbd382-5b09-4fd4-a551-92da84f4bc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:31.498470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1119/2234] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:18:31.502638 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbd382-5b09-4fd4-a551-92da84f4bc8a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:31.512593 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbd382-5b09-4fd4-a551-92da84f4bc8a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:31.517636 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edadf312-a337-417f-bafd-52368ffe0c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:31.518104 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edadf312-a337-417f-bafd-52368ffe0c79 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:31.518390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edadf312-a337-417f-bafd-52368ffe0c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:31.518665 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edadf312-a337-417f-bafd-52368ffe0c79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:31.519401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edadf312-a337-417f-bafd-52368ffe0c79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:18:31.519708 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edadf312-a337-417f-bafd-52368ffe0c79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:31.521312 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edadf312-a337-417f-bafd-52368ffe0c79 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:31.534217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1120/2235] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:18:31.539031 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edadf312-a337-417f-bafd-52368ffe0c79 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:31.549066 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edadf312-a337-417f-bafd-52368ffe0c79 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:31.552436 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3851381-c9b4-4d66-aec9-e09d7456c72f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:31.553627 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b532a-9f98-474b-a39f-fc0d59050728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:31.553873 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3851381-c9b4-4d66-aec9-e09d7456c72f service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:31.554204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b532a-9f98-474b-a39f-fc0d59050728 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:31.554481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b532a-9f98-474b-a39f-fc0d59050728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:31.554758 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b532a-9f98-474b-a39f-fc0d59050728 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:31.555408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b532a-9f98-474b-a39f-fc0d59050728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:31.557865 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b532a-9f98-474b-a39f-fc0d59050728 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:31.564067 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1116/2236] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:18:31.588943 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1121/2237] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:18:31 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:18:31.590078 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3851381-c9b4-4d66-aec9-e09d7456c72f service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:31.602003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3851381-c9b4-4d66-aec9-e09d7456c72f service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:31.622622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:31.663521 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4fe22-dd3c-44f7-a909-0beddb1aa8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:31.664135 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4fe22-dd3c-44f7-a909-0beddb1aa8d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:31.664420 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4fe22-dd3c-44f7-a909-0beddb1aa8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:31.664697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b4fe22-dd3c-44f7-a909-0beddb1aa8d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:31.665331 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4fe22-dd3c-44f7-a909-0beddb1aa8d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:31.667869 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b4fe22-dd3c-44f7-a909-0beddb1aa8d6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:31.695998 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1117/2238] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:18:31 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:31.697636 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b532a-9f98-474b-a39f-fc0d59050728 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:31.704229 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:31.708322 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b532a-9f98-474b-a39f-fc0d59050728 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:31.715552 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d8ab4-76f5-4cc4-b689-192cf35d8700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:31.716412 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d8ab4-76f5-4cc4-b689-192cf35d8700 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:31.716889 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d8ab4-76f5-4cc4-b689-192cf35d8700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:31.717313 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d8ab4-76f5-4cc4-b689-192cf35d8700 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:31.718227 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d8ab4-76f5-4cc4-b689-192cf35d8700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:31.721137 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d8ab4-76f5-4cc4-b689-192cf35d8700 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:31.732434 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1122/2239] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:31.736763 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4fe22-dd3c-44f7-a909-0beddb1aa8d6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:31.748330 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b4fe22-dd3c-44f7-a909-0beddb1aa8d6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:31.753491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0386f35-8873-4f15-854c-6f7f94e909ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:31.754244 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0386f35-8873-4f15-854c-6f7f94e909ca admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:31.754537 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0386f35-8873-4f15-854c-6f7f94e909ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:31.754813 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0386f35-8873-4f15-854c-6f7f94e909ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:31.755490 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0386f35-8873-4f15-854c-6f7f94e909ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:31.757083 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0386f35-8873-4f15-854c-6f7f94e909ca admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:31.770496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1118/2240] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:18:31.771952 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d8ab4-76f5-4cc4-b689-192cf35d8700 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:31.784115 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d8ab4-76f5-4cc4-b689-192cf35d8700 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:31.789776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ee757-a2cd-493f-8f47-39fb6850f04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:31.790509 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ee757-a2cd-493f-8f47-39fb6850f04d admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:31.790797 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ee757-a2cd-493f-8f47-39fb6850f04d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:31.791176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ee757-a2cd-493f-8f47-39fb6850f04d admin admin] PATH_INFO: `/v3/projects/dc518d9b31644a8d8b21cf9c645b1373/users/82da4b8f8bdb4dbab655ff510bfa328f/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:31.811207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ee757-a2cd-493f-8f47-39fb6850f04d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc518d9b31644a8d8b21cf9c645b1373, user_id=82da4b8f8bdb4dbab655ff510bfa328f, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:31.814677 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ee757-a2cd-493f-8f47-39fb6850f04d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:31.830649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1123/2241] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:31 2022] PUT /identity/v3/projects/dc518d9b31644a8d8b21cf9c645b1373/users/82da4b8f8bdb4dbab655ff510bfa328f/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:18:31.835729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a902fc3-156a-4ed5-80ed-8dfa0415b80e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:31.836128 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a902fc3-156a-4ed5-80ed-8dfa0415b80e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:31.836497 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a902fc3-156a-4ed5-80ed-8dfa0415b80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:31.858739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-3a902fc3-156a-4ed5-80ed-8dfa0415b80e None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:31.860004 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3a902fc3-156a-4ed5-80ed-8dfa0415b80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:31.927524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1119/2242] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:18:31.937415 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ee757-a2cd-493f-8f47-39fb6850f04d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:31.939182 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-527ee757-a2cd-493f-8f47-39fb6850f04d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:31.997099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ee757-a2cd-493f-8f47-39fb6850f04d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:32.000785 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16b53c-b256-4cef-a24a-5e54e0b10ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:32.001396 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16b53c-b256-4cef-a24a-5e54e0b10ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:32.001670 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16b53c-b256-4cef-a24a-5e54e0b10ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:32.001936 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16b53c-b256-4cef-a24a-5e54e0b10ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:32.010920 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16b53c-b256-4cef-a24a-5e54e0b10ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:32.012358 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16b53c-b256-4cef-a24a-5e54e0b10ed4 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:32.021490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1124/2243] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:18:32.952223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a902fc3-156a-4ed5-80ed-8dfa0415b80e None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:33.026671 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a902fc3-156a-4ed5-80ed-8dfa0415b80e None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:33.035010 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5516f3c2-dc29-4a1a-9634-25ec70937279 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:33.036246 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516f3c2-dc29-4a1a-9634-25ec70937279 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:33.036722 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516f3c2-dc29-4a1a-9634-25ec70937279 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:33.037206 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516f3c2-dc29-4a1a-9634-25ec70937279 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:33.107759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:33.122468 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5516f3c2-dc29-4a1a-9634-25ec70937279 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:33.125681 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5516f3c2-dc29-4a1a-9634-25ec70937279 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:33.147159 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1120/2244] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:33.872720 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16b53c-b256-4cef-a24a-5e54e0b10ed4 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:33.874326 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6e16b53c-b256-4cef-a24a-5e54e0b10ed4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:33.889814 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16b53c-b256-4cef-a24a-5e54e0b10ed4 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:33.900581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ffcec4-2b62-4397-bb49-c697eda96ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:33.902483 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffcec4-2b62-4397-bb49-c697eda96ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:33.905085 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffcec4-2b62-4397-bb49-c697eda96ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:33.905836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffcec4-2b62-4397-bb49-c697eda96ec3 admin admin] PATH_INFO: `/v3/users/8975087c65094417a5b7068c89128ed4` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:33.909408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ffcec4-2b62-4397-bb49-c697eda96ec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8975087c65094417a5b7068c89128ed4)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:33.912445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ffcec4-2b62-4397-bb49-c697eda96ec3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:33.982259 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-93ffcec4-2b62-4397-bb49-c697eda96ec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8975087c65094417a5b7068c89128ed4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:33.989411 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-93ffcec4-2b62-4397-bb49-c697eda96ec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8975087c65094417a5b7068c89128ed4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:33.990183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-93ffcec4-2b62-4397-bb49-c697eda96ec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8975087c65094417a5b7068c89128ed4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:34.000208 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-93ffcec4-2b62-4397-bb49-c697eda96ec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8975087c65094417a5b7068c89128ed4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:34.015689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-93ffcec4-2b62-4397-bb49-c697eda96ec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8975087c65094417a5b7068c89128ed4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:34.022194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1125/2245] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:18:33 2022] DELETE /identity/v3/users/8975087c65094417a5b7068c89128ed4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:34.027080 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74853dfe-85a0-4127-9ca0-64a71b7e353a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:34.027557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74853dfe-85a0-4127-9ca0-64a71b7e353a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:34.027668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74853dfe-85a0-4127-9ca0-64a71b7e353a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:34.054829 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-74853dfe-85a0-4127-9ca0-64a71b7e353a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:34.056688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-74853dfe-85a0-4127-9ca0-64a71b7e353a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:34.144088 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1121/2246] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:34.158378 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ffcec4-2b62-4397-bb49-c697eda96ec3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:34.161892 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-93ffcec4-2b62-4397-bb49-c697eda96ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:34.292340 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:34.308274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ffcec4-2b62-4397-bb49-c697eda96ec3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:34.328006 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b74b9-1fba-44aa-828e-27a46b158536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:34.332796 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b74b9-1fba-44aa-828e-27a46b158536 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:34.333159 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b74b9-1fba-44aa-828e-27a46b158536 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:34.335720 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b74b9-1fba-44aa-828e-27a46b158536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:34.350038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b74b9-1fba-44aa-828e-27a46b158536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:34.352446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b74b9-1fba-44aa-828e-27a46b158536 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:34.371613 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1126/2247] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:35.085433 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74853dfe-85a0-4127-9ca0-64a71b7e353a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:35.087083 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-74853dfe-85a0-4127-9ca0-64a71b7e353a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:35.101093 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74853dfe-85a0-4127-9ca0-64a71b7e353a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:35.108211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d41b7-1b94-486e-8c73-c48ce5391362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:35.109183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d41b7-1b94-486e-8c73-c48ce5391362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:35.109717 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d41b7-1b94-486e-8c73-c48ce5391362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:35.110222 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d41b7-1b94-486e-8c73-c48ce5391362 admin admin] PATH_INFO: `/v3/projects/4f9bd622b30c4e3cb47ff332452c63e1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:35.111960 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d41b7-1b94-486e-8c73-c48ce5391362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f9bd622b30c4e3cb47ff332452c63e1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:35.114965 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d41b7-1b94-486e-8c73-c48ce5391362 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:35.210077 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf3d41b7-1b94-486e-8c73-c48ce5391362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f9bd622b30c4e3cb47ff332452c63e1', 'request_id': 'req-cf3d41b7-1b94-486e-8c73-c48ce5391362'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:35.225477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf3d41b7-1b94-486e-8c73-c48ce5391362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f9bd622b30c4e3cb47ff332452c63e1', 'request_id': 'req-cf3d41b7-1b94-486e-8c73-c48ce5391362'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:35.236536 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf3d41b7-1b94-486e-8c73-c48ce5391362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f9bd622b30c4e3cb47ff332452c63e1', 'request_id': 'req-cf3d41b7-1b94-486e-8c73-c48ce5391362'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:35.264722 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf3d41b7-1b94-486e-8c73-c48ce5391362 admin admin] The token cache is being invalidate because project 4f9bd622b30c4e3cb47ff332452c63e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:18:35.265283 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf3d41b7-1b94-486e-8c73-c48ce5391362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:35.267143 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cf3d41b7-1b94-486e-8c73-c48ce5391362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:35.271272 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1122/2248] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:18:35 2022] DELETE /identity/v3/projects/4f9bd622b30c4e3cb47ff332452c63e1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:35.277863 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b74b9-1fba-44aa-828e-27a46b158536 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:35.298566 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b74b9-1fba-44aa-828e-27a46b158536 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:35.416702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f1625a-5da7-46f1-b036-d25e3febddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:35.416702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1625a-5da7-46f1-b036-d25e3febddad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:35.416998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1625a-5da7-46f1-b036-d25e3febddad admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:35.417078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f1625a-5da7-46f1-b036-d25e3febddad admin admin] PATH_INFO: `/v3/users/ccf5a72fe97c401f8e8349a027488f75` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:35.420648 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f1625a-5da7-46f1-b036-d25e3febddad admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccf5a72fe97c401f8e8349a027488f75)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:35.424448 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f1625a-5da7-46f1-b036-d25e3febddad admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:35.520952 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-49f1625a-5da7-46f1-b036-d25e3febddad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccf5a72fe97c401f8e8349a027488f75'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:35.537999 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-49f1625a-5da7-46f1-b036-d25e3febddad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccf5a72fe97c401f8e8349a027488f75'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:35.539910 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-49f1625a-5da7-46f1-b036-d25e3febddad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccf5a72fe97c401f8e8349a027488f75'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:35.558341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-49f1625a-5da7-46f1-b036-d25e3febddad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccf5a72fe97c401f8e8349a027488f75'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:35.581386 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-49f1625a-5da7-46f1-b036-d25e3febddad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccf5a72fe97c401f8e8349a027488f75'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:35.589802 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1127/2249] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:18:35 2022] DELETE /identity/v3/users/ccf5a72fe97c401f8e8349a027488f75 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:36.486946 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d41b7-1b94-486e-8c73-c48ce5391362 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:36.489660 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cf3d41b7-1b94-486e-8c73-c48ce5391362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:36.518799 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d41b7-1b94-486e-8c73-c48ce5391362 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:36.631473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e78b7-0e86-443e-8041-42464cec5bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:36.632508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e78b7-0e86-443e-8041-42464cec5bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:36.632980 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e78b7-0e86-443e-8041-42464cec5bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:36.633445 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e78b7-0e86-443e-8041-42464cec5bfb admin admin] PATH_INFO: `/v3/projects/9c18d84f2a3647e48d8b4d10ec3f36d2` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:36.635538 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e78b7-0e86-443e-8041-42464cec5bfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c18d84f2a3647e48d8b4d10ec3f36d2)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:36.641350 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e78b7-0e86-443e-8041-42464cec5bfb admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:36.678765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:36.730683 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-826e78b7-0e86-443e-8041-42464cec5bfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c18d84f2a3647e48d8b4d10ec3f36d2', 'request_id': 'req-826e78b7-0e86-443e-8041-42464cec5bfb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:36.739314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-826e78b7-0e86-443e-8041-42464cec5bfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c18d84f2a3647e48d8b4d10ec3f36d2', 'request_id': 'req-826e78b7-0e86-443e-8041-42464cec5bfb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:36.750640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-826e78b7-0e86-443e-8041-42464cec5bfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c18d84f2a3647e48d8b4d10ec3f36d2', 'request_id': 'req-826e78b7-0e86-443e-8041-42464cec5bfb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:36.773378 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-826e78b7-0e86-443e-8041-42464cec5bfb admin admin] The token cache is being invalidate because project 9c18d84f2a3647e48d8b4d10ec3f36d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:18:36.774030 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-826e78b7-0e86-443e-8041-42464cec5bfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:36.775521 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-826e78b7-0e86-443e-8041-42464cec5bfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:36.778461 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1123/2250] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:18:36 2022] DELETE /identity/v3/projects/9c18d84f2a3647e48d8b4d10ec3f36d2 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:36.783375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a00be9-611b-4760-a9eb-f488be837c24 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:36.783785 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a00be9-611b-4760-a9eb-f488be837c24 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:36.783932 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a00be9-611b-4760-a9eb-f488be837c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:36.806538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c3a00be9-611b-4760-a9eb-f488be837c24 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:36.808308 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c3a00be9-611b-4760-a9eb-f488be837c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:36.897620 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1128/2251] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:36.906297 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48073f-fd78-4d65-b366-31cdb39caa3a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:36.906542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48073f-fd78-4d65-b366-31cdb39caa3a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:36.906774 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48073f-fd78-4d65-b366-31cdb39caa3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:36.929668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9a48073f-fd78-4d65-b366-31cdb39caa3a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:36.930915 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9a48073f-fd78-4d65-b366-31cdb39caa3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:36.947557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1124/2252] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:36.952677 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a00be9-611b-4760-a9eb-f488be837c24 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:36.964792 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a00be9-611b-4760-a9eb-f488be837c24 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:36.970779 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2723bf7-4be7-48c3-8986-024fb96434a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:36.971709 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2723bf7-4be7-48c3-8986-024fb96434a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:36.971933 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2723bf7-4be7-48c3-8986-024fb96434a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:36.972435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2723bf7-4be7-48c3-8986-024fb96434a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:36.973347 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2723bf7-4be7-48c3-8986-024fb96434a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:18:36.973661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2723bf7-4be7-48c3-8986-024fb96434a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:36.975363 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2723bf7-4be7-48c3-8986-024fb96434a0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:36.978457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:36.994043 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1129/2253] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:18:36.998610 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48073f-fd78-4d65-b366-31cdb39caa3a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:37.009907 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48073f-fd78-4d65-b366-31cdb39caa3a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:37.015602 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe3050-3e7f-46a3-a176-7336b4186682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:37.016236 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe3050-3e7f-46a3-a176-7336b4186682 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:37.016506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe3050-3e7f-46a3-a176-7336b4186682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:37.016790 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe3050-3e7f-46a3-a176-7336b4186682 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:37.017437 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe3050-3e7f-46a3-a176-7336b4186682 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:37.019411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe3050-3e7f-46a3-a176-7336b4186682 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:37.068743 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1125/2254] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:18:36 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:37.073594 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2723bf7-4be7-48c3-8986-024fb96434a0 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:37.084319 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2723bf7-4be7-48c3-8986-024fb96434a0 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:37.156654 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a4c09-23a5-4e89-ae3f-3e05e7697f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:37.157498 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a4c09-23a5-4e89-ae3f-3e05e7697f86 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:37.157959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a4c09-23a5-4e89-ae3f-3e05e7697f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:37.158397 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a4c09-23a5-4e89-ae3f-3e05e7697f86 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:37.159460 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a4c09-23a5-4e89-ae3f-3e05e7697f86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:37.161847 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a4c09-23a5-4e89-ae3f-3e05e7697f86 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:37.197380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1130/2255] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:18:37 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:37.201780 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe3050-3e7f-46a3-a176-7336b4186682 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:37.217193 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe3050-3e7f-46a3-a176-7336b4186682 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:37.223170 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24499782-0e03-4ead-bbe4-defa31778f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:37.224113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24499782-0e03-4ead-bbe4-defa31778f20 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:37.224604 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24499782-0e03-4ead-bbe4-defa31778f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:37.225262 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24499782-0e03-4ead-bbe4-defa31778f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:37.226074 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24499782-0e03-4ead-bbe4-defa31778f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:37.227936 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24499782-0e03-4ead-bbe4-defa31778f20 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:37.241416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1126/2256] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:37.245252 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a4c09-23a5-4e89-ae3f-3e05e7697f86 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:37.258789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a4c09-23a5-4e89-ae3f-3e05e7697f86 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:37.265422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0900e505-0789-42db-acec-a47a837843ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:37.266502 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900e505-0789-42db-acec-a47a837843ba admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:37.266789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900e505-0789-42db-acec-a47a837843ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:37.267121 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0900e505-0789-42db-acec-a47a837843ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:37.267787 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900e505-0789-42db-acec-a47a837843ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:37.269487 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0900e505-0789-42db-acec-a47a837843ba admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:37.282895 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1131/2257] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:37.287958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24499782-0e03-4ead-bbe4-defa31778f20 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:37.312179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24499782-0e03-4ead-bbe4-defa31778f20 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:37.317612 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25481a-edde-4233-9f3f-14392d3b9a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:37.318313 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25481a-edde-4233-9f3f-14392d3b9a1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:37.318918 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25481a-edde-4233-9f3f-14392d3b9a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:37.318918 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25481a-edde-4233-9f3f-14392d3b9a1a admin admin] PATH_INFO: `/v3/projects/a8cad99a1f044f75824cc3c356308002/users/d60a98fb145f497c8b4976c1522aa96a/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:37.356936 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25481a-edde-4233-9f3f-14392d3b9a1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8cad99a1f044f75824cc3c356308002, user_id=d60a98fb145f497c8b4976c1522aa96a, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:37.362369 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25481a-edde-4233-9f3f-14392d3b9a1a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:37.398403 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1127/2258] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:37 2022] PUT /identity/v3/projects/a8cad99a1f044f75824cc3c356308002/users/d60a98fb145f497c8b4976c1522aa96a/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:37.412035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae798ee5-692f-428b-9911-98303c6612e5 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:37.413924 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae798ee5-692f-428b-9911-98303c6612e5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:37.415110 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae798ee5-692f-428b-9911-98303c6612e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:37.494222 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049422e-1648-4cf9-928c-084eb205f572 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:37.495586 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049422e-1648-4cf9-928c-084eb205f572 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:37.496049 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049422e-1648-4cf9-928c-084eb205f572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:37.517383 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-ae798ee5-692f-428b-9911-98303c6612e5 None None] MFA Rules not processed for user `d60a98fb145f497c8b4976c1522aa96a`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:37.517383 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ae798ee5-692f-428b-9911-98303c6612e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:37.533909 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-c049422e-1648-4cf9-928c-084eb205f572 None None] MFA Rules not processed for user `82da4b8f8bdb4dbab655ff510bfa328f`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:37.535388 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c049422e-1648-4cf9-928c-084eb205f572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:37.681010 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1128/2259] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:37.684718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049422e-1648-4cf9-928c-084eb205f572 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:37.686191 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1132/2260] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:37.687035 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c049422e-1648-4cf9-928c-084eb205f572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:37.745200 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:37.762003 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae798ee5-692f-428b-9911-98303c6612e5 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:37.764366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ae798ee5-692f-428b-9911-98303c6612e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:37.807945 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049422e-1648-4cf9-928c-084eb205f572 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:37.819533 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd698f-9f8b-499c-b631-720f4b3c36ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:37.821017 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd698f-9f8b-499c-b631-720f4b3c36ba admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:37.821695 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd698f-9f8b-499c-b631-720f4b3c36ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:37.821882 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd698f-9f8b-499c-b631-720f4b3c36ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:37.823651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd698f-9f8b-499c-b631-720f4b3c36ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:37.826450 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd698f-9f8b-499c-b631-720f4b3c36ba admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:37.867708 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1129/2261] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:18:37 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:37.868464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd698f-9f8b-499c-b631-720f4b3c36ba admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:37.899004 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae798ee5-692f-428b-9911-98303c6612e5 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:37.946201 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:37.963042 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd698f-9f8b-499c-b631-720f4b3c36ba admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:37.978126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f0cd3-ce03-4af7-a12b-eb7c3a6962e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:37.978126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f0cd3-ce03-4af7-a12b-eb7c3a6962e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:37.978126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f0cd3-ce03-4af7-a12b-eb7c3a6962e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:37.978603 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f0cd3-ce03-4af7-a12b-eb7c3a6962e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:37.979420 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f0cd3-ce03-4af7-a12b-eb7c3a6962e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:37.982296 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f0cd3-ce03-4af7-a12b-eb7c3a6962e8 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:38.017130 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dbed9-490a-40e5-b466-5d5f331ea7ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:38.017862 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dbed9-490a-40e5-b466-5d5f331ea7ad service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:38.018146 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dbed9-490a-40e5-b466-5d5f331ea7ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:38.018418 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dbed9-490a-40e5-b466-5d5f331ea7ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:38.066100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1130/2262] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:18:37 2022] POST /identity/v3/users => generated 299 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:38.069099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f0cd3-ce03-4af7-a12b-eb7c3a6962e8 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:38.092425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f0cd3-ce03-4af7-a12b-eb7c3a6962e8 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:38.098758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd405c8-f173-4ada-bcf9-bde6f86e22ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:38.098758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd405c8-f173-4ada-bcf9-bde6f86e22ac admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:38.098758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd405c8-f173-4ada-bcf9-bde6f86e22ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:38.098758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd405c8-f173-4ada-bcf9-bde6f86e22ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:38.099322 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd405c8-f173-4ada-bcf9-bde6f86e22ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:38.101155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd405c8-f173-4ada-bcf9-bde6f86e22ac admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:38.115737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1131/2263] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:38.120748 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd405c8-f173-4ada-bcf9-bde6f86e22ac admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:38.140056 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd405c8-f173-4ada-bcf9-bde6f86e22ac admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:38.149988 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8a278-05b6-4b0e-803d-d751e421bbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:38.150801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8a278-05b6-4b0e-803d-d751e421bbb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:38.151317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8a278-05b6-4b0e-803d-d751e421bbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:38.151683 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8a278-05b6-4b0e-803d-d751e421bbb2 admin admin] PATH_INFO: `/v3/projects/83ef05f3398644eda7a2932e00314109/users/c85ec9988c5b4002bab7d2267a305593/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:38.154097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634dbed9-490a-40e5-b466-5d5f331ea7ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:38.156402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634dbed9-490a-40e5-b466-5d5f331ea7ad service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:38.170627 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1133/2264] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:38.171719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dbed9-490a-40e5-b466-5d5f331ea7ad service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:38.172986 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8a278-05b6-4b0e-803d-d751e421bbb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83ef05f3398644eda7a2932e00314109, user_id=c85ec9988c5b4002bab7d2267a305593, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:38.173925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-634dbed9-490a-40e5-b466-5d5f331ea7ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:38.176133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8a278-05b6-4b0e-803d-d751e421bbb2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:38.195258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1132/2265] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:38 2022] PUT /identity/v3/projects/83ef05f3398644eda7a2932e00314109/users/c85ec9988c5b4002bab7d2267a305593/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:38.199383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8a278-05b6-4b0e-803d-d751e421bbb2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:38.247211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dbed9-490a-40e5-b466-5d5f331ea7ad service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:38.270296 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8a278-05b6-4b0e-803d-d751e421bbb2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:38.275610 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6f5ff-a878-4c75-941e-0a15e0b5baa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:38.276267 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6f5ff-a878-4c75-941e-0a15e0b5baa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:38.276540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6f5ff-a878-4c75-941e-0a15e0b5baa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:38.276810 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6f5ff-a878-4c75-941e-0a15e0b5baa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:38.277454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6f5ff-a878-4c75-941e-0a15e0b5baa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:38.279115 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff6f5ff-a878-4c75-941e-0a15e0b5baa1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:38.293318 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1133/2266] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:18:38.298820 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6f5ff-a878-4c75-941e-0a15e0b5baa1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:38.310313 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff6f5ff-a878-4c75-941e-0a15e0b5baa1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:38.311275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9509f1-29a7-4432-a872-5e1393c6022f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:38.311915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9509f1-29a7-4432-a872-5e1393c6022f service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:38.312295 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9509f1-29a7-4432-a872-5e1393c6022f service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:38.312678 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9509f1-29a7-4432-a872-5e1393c6022f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:38.314990 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-af9509f1-29a7-4432-a872-5e1393c6022f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:38.316263 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0384f99-095f-4f49-a38d-6ca0aeb28e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:38.317031 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0384f99-095f-4f49-a38d-6ca0aeb28e29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:38.317372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0384f99-095f-4f49-a38d-6ca0aeb28e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:38.317715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0384f99-095f-4f49-a38d-6ca0aeb28e29 admin admin] PATH_INFO: `/v3/projects/83ef05f3398644eda7a2932e00314109/users/c85ec9988c5b4002bab7d2267a305593/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:38.320249 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0384f99-095f-4f49-a38d-6ca0aeb28e29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83ef05f3398644eda7a2932e00314109, user_id=c85ec9988c5b4002bab7d2267a305593, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:38.323881 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0384f99-095f-4f49-a38d-6ca0aeb28e29 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:38.326554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:38.332402 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9509f1-29a7-4432-a872-5e1393c6022f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:38.335235 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9509f1-29a7-4432-a872-5e1393c6022f service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:38.336745 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1134/2267] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:38 2022] PUT /identity/v3/projects/83ef05f3398644eda7a2932e00314109/users/c85ec9988c5b4002bab7d2267a305593/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:38.347915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1134/2268] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:43.074017 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0384f99-095f-4f49-a38d-6ca0aeb28e29 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:43.075853 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c0384f99-095f-4f49-a38d-6ca0aeb28e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:43.233842 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0384f99-095f-4f49-a38d-6ca0aeb28e29 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:43.251127 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6685e-a304-4c9a-b0ab-177ddcc3a470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:43.252932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6685e-a304-4c9a-b0ab-177ddcc3a470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:43.253294 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6685e-a304-4c9a-b0ab-177ddcc3a470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:43.253642 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6685e-a304-4c9a-b0ab-177ddcc3a470 admin admin] PATH_INFO: `/v3/users/37e6d36d529146299970874660e92e43` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:43.255281 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6685e-a304-4c9a-b0ab-177ddcc3a470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37e6d36d529146299970874660e92e43)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:43.258151 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b6685e-a304-4c9a-b0ab-177ddcc3a470 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:43.364549 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e1b6685e-a304-4c9a-b0ab-177ddcc3a470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37e6d36d529146299970874660e92e43'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:43.379134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e1b6685e-a304-4c9a-b0ab-177ddcc3a470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37e6d36d529146299970874660e92e43'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:43.379980 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e1b6685e-a304-4c9a-b0ab-177ddcc3a470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37e6d36d529146299970874660e92e43'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:43.405368 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e1b6685e-a304-4c9a-b0ab-177ddcc3a470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37e6d36d529146299970874660e92e43'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:43.425476 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-e1b6685e-a304-4c9a-b0ab-177ddcc3a470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37e6d36d529146299970874660e92e43'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:43.441951 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1135/2269] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:18:43 2022] DELETE /identity/v3/users/37e6d36d529146299970874660e92e43 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:43.924404 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9509f1-29a7-4432-a872-5e1393c6022f service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:43.945784 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9509f1-29a7-4432-a872-5e1393c6022f service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:44.006567 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:44.042624 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b113b-2cfc-4481-a35e-685500623794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:44.043521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b113b-2cfc-4481-a35e-685500623794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:44.043879 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b113b-2cfc-4481-a35e-685500623794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:44.044230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b113b-2cfc-4481-a35e-685500623794 admin admin] PATH_INFO: `/v3/projects/36f63b28bb4f4654b53a01507b337d52` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:44.045490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b113b-2cfc-4481-a35e-685500623794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36f63b28bb4f4654b53a01507b337d52)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:44.047725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b113b-2cfc-4481-a35e-685500623794 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:44.130166 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7b7b113b-2cfc-4481-a35e-685500623794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36f63b28bb4f4654b53a01507b337d52', 'request_id': 'req-7b7b113b-2cfc-4481-a35e-685500623794'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:44.141538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7b7b113b-2cfc-4481-a35e-685500623794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36f63b28bb4f4654b53a01507b337d52', 'request_id': 'req-7b7b113b-2cfc-4481-a35e-685500623794'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:44.148361 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7b7b113b-2cfc-4481-a35e-685500623794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36f63b28bb4f4654b53a01507b337d52', 'request_id': 'req-7b7b113b-2cfc-4481-a35e-685500623794'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:44.171538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7b7b113b-2cfc-4481-a35e-685500623794 admin admin] The token cache is being invalidate because project 36f63b28bb4f4654b53a01507b337d52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:18:44.171787 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7b7b113b-2cfc-4481-a35e-685500623794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:44.172563 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7b7b113b-2cfc-4481-a35e-685500623794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:44.174183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1135/2270] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:18:43 2022] DELETE /identity/v3/projects/36f63b28bb4f4654b53a01507b337d52 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:44.179164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd486c2a-3443-4f53-bf16-1d8af11d7d43 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:44.179646 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd486c2a-3443-4f53-bf16-1d8af11d7d43 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:44.179757 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd486c2a-3443-4f53-bf16-1d8af11d7d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:44.204887 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-fd486c2a-3443-4f53-bf16-1d8af11d7d43 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:44.206936 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fd486c2a-3443-4f53-bf16-1d8af11d7d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:44.321989 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1136/2271] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:44.329648 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1112b6-dfd0-455e-a201-de26a23de6fb None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:44.330003 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1112b6-dfd0-455e-a201-de26a23de6fb None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:44.330208 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1112b6-dfd0-455e-a201-de26a23de6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:44.365339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-4a1112b6-dfd0-455e-a201-de26a23de6fb None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:44.366717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4a1112b6-dfd0-455e-a201-de26a23de6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:44.380962 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1136/2272] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:44.381656 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd486c2a-3443-4f53-bf16-1d8af11d7d43 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:44.397762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd486c2a-3443-4f53-bf16-1d8af11d7d43 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:44.403888 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdba365-9b82-4444-a42e-df3b496c479a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:44.404681 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdba365-9b82-4444-a42e-df3b496c479a admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:44.405069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdba365-9b82-4444-a42e-df3b496c479a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:44.405491 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdba365-9b82-4444-a42e-df3b496c479a admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:44.406241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdba365-9b82-4444-a42e-df3b496c479a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:18:44.406655 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdba365-9b82-4444-a42e-df3b496c479a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:44.408625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdba365-9b82-4444-a42e-df3b496c479a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:44.419966 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:44.428893 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1112b6-dfd0-455e-a201-de26a23de6fb None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:44.433915 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1137/2273] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:18:44.439843 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1112b6-dfd0-455e-a201-de26a23de6fb None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:44.445007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638e51b-582d-449e-81b1-376fbc830fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:44.445647 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638e51b-582d-449e-81b1-376fbc830fec admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:44.445931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638e51b-582d-449e-81b1-376fbc830fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:44.446208 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638e51b-582d-449e-81b1-376fbc830fec admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:44.446886 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638e51b-582d-449e-81b1-376fbc830fec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:44.449718 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638e51b-582d-449e-81b1-376fbc830fec admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:44.488382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1137/2274] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:18:44 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:44.493646 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdba365-9b82-4444-a42e-df3b496c479a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:44.505183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdba365-9b82-4444-a42e-df3b496c479a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:44.612121 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed24ad-fb6f-45ba-9e12-1df3705b6dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:44.612807 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed24ad-fb6f-45ba-9e12-1df3705b6dec admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:44.613095 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed24ad-fb6f-45ba-9e12-1df3705b6dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:44.613370 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed24ad-fb6f-45ba-9e12-1df3705b6dec admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:44.614025 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed24ad-fb6f-45ba-9e12-1df3705b6dec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:44.616017 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed24ad-fb6f-45ba-9e12-1df3705b6dec admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:44.642543 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1138/2275] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:18:44 2022] POST /identity/v3/users => generated 284 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:44.646788 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638e51b-582d-449e-81b1-376fbc830fec admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:44.669016 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638e51b-582d-449e-81b1-376fbc830fec admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:44.688766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715dc418-298b-455a-8423-1a978a08113f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:44.689400 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dc418-298b-455a-8423-1a978a08113f admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:44.689686 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dc418-298b-455a-8423-1a978a08113f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:44.689965 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dc418-298b-455a-8423-1a978a08113f admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:44.690633 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715dc418-298b-455a-8423-1a978a08113f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:44.693926 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715dc418-298b-455a-8423-1a978a08113f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:44.717905 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1138/2276] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:44.718309 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed24ad-fb6f-45ba-9e12-1df3705b6dec admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:44.764905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed24ad-fb6f-45ba-9e12-1df3705b6dec admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:44.771737 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a7e91-7d08-4ba9-ac8b-13607bf3ce84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:44.772223 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a7e91-7d08-4ba9-ac8b-13607bf3ce84 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:44.772513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a7e91-7d08-4ba9-ac8b-13607bf3ce84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:44.772864 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a7e91-7d08-4ba9-ac8b-13607bf3ce84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:44.775145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a7e91-7d08-4ba9-ac8b-13607bf3ce84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:44.778929 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a7e91-7d08-4ba9-ac8b-13607bf3ce84 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:44.801012 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1139/2277] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:18:44.805514 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715dc418-298b-455a-8423-1a978a08113f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:44.817668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715dc418-298b-455a-8423-1a978a08113f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:44.837299 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0aa49-6dd9-47c9-ad4d-0259c71accd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:44.838002 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0aa49-6dd9-47c9-ad4d-0259c71accd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:44.838918 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0aa49-6dd9-47c9-ad4d-0259c71accd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:44.839216 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0aa49-6dd9-47c9-ad4d-0259c71accd0 admin admin] PATH_INFO: `/v3/projects/dc478db2bb82408194391f5e0165d7b0/users/dcda6d1c205d4ec0bd4adcd65fa25248/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:44.861048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0aa49-6dd9-47c9-ad4d-0259c71accd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc478db2bb82408194391f5e0165d7b0, user_id=dcda6d1c205d4ec0bd4adcd65fa25248, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:44.864891 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0aa49-6dd9-47c9-ad4d-0259c71accd0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:44.892925 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1139/2278] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:44 2022] PUT /identity/v3/projects/dc478db2bb82408194391f5e0165d7b0/users/dcda6d1c205d4ec0bd4adcd65fa25248/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:44.894419 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffd1b2-c379-4bb8-aa8a-48c717f19e8c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:44.894929 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffd1b2-c379-4bb8-aa8a-48c717f19e8c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:44.895236 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffd1b2-c379-4bb8-aa8a-48c717f19e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:44.953688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a2ffd1b2-c379-4bb8-aa8a-48c717f19e8c None None] MFA Rules not processed for user `dcda6d1c205d4ec0bd4adcd65fa25248`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:44.955137 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a2ffd1b2-c379-4bb8-aa8a-48c717f19e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:45.049534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1140/2279] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:45.057593 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0aa49-6dd9-47c9-ad4d-0259c71accd0 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:45.063527 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b7a0aa49-6dd9-47c9-ad4d-0259c71accd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:45.144815 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0aa49-6dd9-47c9-ad4d-0259c71accd0 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:45.159408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff9fa0-fe65-46bf-906a-7936d313c4e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:45.159408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff9fa0-fe65-46bf-906a-7936d313c4e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:45.159408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff9fa0-fe65-46bf-906a-7936d313c4e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:45.159408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ff9fa0-fe65-46bf-906a-7936d313c4e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:45.162931 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6998f74-e8b2-4dac-a6ab-f9f2bcbd2912 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:45.163843 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6998f74-e8b2-4dac-a6ab-f9f2bcbd2912 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:45.164254 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6998f74-e8b2-4dac-a6ab-f9f2bcbd2912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:45.172088 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff9fa0-fe65-46bf-906a-7936d313c4e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:45.173608 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ff9fa0-fe65-46bf-906a-7936d313c4e8 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:45.180192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:45.192244 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1140/2280] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:45.202699 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-b6998f74-e8b2-4dac-a6ab-f9f2bcbd2912 None None] MFA Rules not processed for user `c85ec9988c5b4002bab7d2267a305593`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:45.203971 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b6998f74-e8b2-4dac-a6ab-f9f2bcbd2912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:45.280484 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:45.299096 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1141/2281] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:18:45.309358 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff9fa0-fe65-46bf-906a-7936d313c4e8 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:45.310841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-68ff9fa0-fe65-46bf-906a-7936d313c4e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:45.395797 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ff9fa0-fe65-46bf-906a-7936d313c4e8 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:45.399830 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f51a4-5254-459b-8b3c-58bb1b280102 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8aa29639a3143d2a270e3248d2205d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:45.400558 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f51a4-5254-459b-8b3c-58bb1b280102 service glance] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:45.401497 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f51a4-5254-459b-8b3c-58bb1b280102 service glance] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:45.401864 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f51a4-5254-459b-8b3c-58bb1b280102 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:45.403899 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6d4f51a4-5254-459b-8b3c-58bb1b280102 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:45.513621 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f51a4-5254-459b-8b3c-58bb1b280102 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:45.515395 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f51a4-5254-459b-8b3c-58bb1b280102 service glance] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:45.530226 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1141/2282] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:45.702766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd7bce-34d8-4f2c-970a-617d748f66b9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:45.703396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd7bce-34d8-4f2c-970a-617d748f66b9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:45.703804 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd7bce-34d8-4f2c-970a-617d748f66b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:45.726004 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-69bd7bce-34d8-4f2c-970a-617d748f66b9 None None] MFA Rules not processed for user `280d47ed79f14105a83b18b2cac58b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:45.727414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-69bd7bce-34d8-4f2c-970a-617d748f66b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:45.793293 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1142/2283] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Oct 17 03:18:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:45.816632 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f51a4-5254-459b-8b3c-58bb1b280102 service glance] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:45.818046 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6d4f51a4-5254-459b-8b3c-58bb1b280102 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:45.905613 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f51a4-5254-459b-8b3c-58bb1b280102 service glance] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:45.910680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f99cc-65df-4df8-82ab-d3d5c5836ba8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eceda8b3a8ba4233851d0138d8f4d06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:45.911665 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f99cc-65df-4df8-82ab-d3d5c5836ba8 service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:45.911952 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f99cc-65df-4df8-82ab-d3d5c5836ba8 service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:45.912259 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f99cc-65df-4df8-82ab-d3d5c5836ba8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:45.921295 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f99cc-65df-4df8-82ab-d3d5c5836ba8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:45.923031 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f99cc-65df-4df8-82ab-d3d5c5836ba8 service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:45.931419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1142/2284] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Oct 17 03:18:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 17 03:18:51.701147 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd7bce-34d8-4f2c-970a-617d748f66b9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:51.707199 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-69bd7bce-34d8-4f2c-970a-617d748f66b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:51.876280 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd7bce-34d8-4f2c-970a-617d748f66b9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:51.883509 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d137c3e-a739-4ee4-963a-e150e593be7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:51.884465 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d137c3e-a739-4ee4-963a-e150e593be7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:51.884848 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d137c3e-a739-4ee4-963a-e150e593be7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:51.885438 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d137c3e-a739-4ee4-963a-e150e593be7f admin admin] PATH_INFO: `/v3/users/16a5c17cd80c497d8453bcadceb091c4` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:51.887145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d137c3e-a739-4ee4-963a-e150e593be7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=16a5c17cd80c497d8453bcadceb091c4)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:51.910122 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d137c3e-a739-4ee4-963a-e150e593be7f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:52.012285 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5d137c3e-a739-4ee4-963a-e150e593be7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16a5c17cd80c497d8453bcadceb091c4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:52.021819 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5d137c3e-a739-4ee4-963a-e150e593be7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16a5c17cd80c497d8453bcadceb091c4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:52.022663 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5d137c3e-a739-4ee4-963a-e150e593be7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16a5c17cd80c497d8453bcadceb091c4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:52.035353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5d137c3e-a739-4ee4-963a-e150e593be7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16a5c17cd80c497d8453bcadceb091c4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:52.053161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-5d137c3e-a739-4ee4-963a-e150e593be7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16a5c17cd80c497d8453bcadceb091c4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:52.060913 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1143/2285] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:18:51 2022] DELETE /identity/v3/users/16a5c17cd80c497d8453bcadceb091c4 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:52.323506 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f99cc-65df-4df8-82ab-d3d5c5836ba8 service swift] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:52.362003 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f99cc-65df-4df8-82ab-d3d5c5836ba8 service swift] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:52.423353 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:52.443401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33f906-8274-4dfa-9c3a-df9e94c48847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:52.444437 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33f906-8274-4dfa-9c3a-df9e94c48847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:52.444921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33f906-8274-4dfa-9c3a-df9e94c48847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:52.445929 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33f906-8274-4dfa-9c3a-df9e94c48847 admin admin] PATH_INFO: `/v3/projects/3836eae073a146a796fd7da052ed45cd` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:52.450483 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33f906-8274-4dfa-9c3a-df9e94c48847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3836eae073a146a796fd7da052ed45cd)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:52.452447 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33f906-8274-4dfa-9c3a-df9e94c48847 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:52.514481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1c33f906-8274-4dfa-9c3a-df9e94c48847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3836eae073a146a796fd7da052ed45cd', 'request_id': 'req-1c33f906-8274-4dfa-9c3a-df9e94c48847'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:52.520419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1c33f906-8274-4dfa-9c3a-df9e94c48847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3836eae073a146a796fd7da052ed45cd', 'request_id': 'req-1c33f906-8274-4dfa-9c3a-df9e94c48847'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:52.531880 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1c33f906-8274-4dfa-9c3a-df9e94c48847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3836eae073a146a796fd7da052ed45cd', 'request_id': 'req-1c33f906-8274-4dfa-9c3a-df9e94c48847'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:52.544897 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1c33f906-8274-4dfa-9c3a-df9e94c48847 admin admin] The token cache is being invalidate because project 3836eae073a146a796fd7da052ed45cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:18:52.545305 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1c33f906-8274-4dfa-9c3a-df9e94c48847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:52.546007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1c33f906-8274-4dfa-9c3a-df9e94c48847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:52.547903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1143/2286] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:18:52 2022] DELETE /identity/v3/projects/3836eae073a146a796fd7da052ed45cd => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:52.551978 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d137c3e-a739-4ee4-963a-e150e593be7f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:52.553345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5d137c3e-a739-4ee4-963a-e150e593be7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:52.571241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d137c3e-a739-4ee4-963a-e150e593be7f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:52.585544 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:52.641445 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04156da5-fb87-4e96-87e6-d2af43b35ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:52.642144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04156da5-fb87-4e96-87e6-d2af43b35ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:52.642426 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04156da5-fb87-4e96-87e6-d2af43b35ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:52.642707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04156da5-fb87-4e96-87e6-d2af43b35ba8 admin admin] PATH_INFO: `/v3/users/9655f3864c2f42b2b113bf372e92e47c` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:52.643911 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04156da5-fb87-4e96-87e6-d2af43b35ba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9655f3864c2f42b2b113bf372e92e47c)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:52.646007 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04156da5-fb87-4e96-87e6-d2af43b35ba8 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:52.704500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-04156da5-fb87-4e96-87e6-d2af43b35ba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9655f3864c2f42b2b113bf372e92e47c'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:52.714658 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-04156da5-fb87-4e96-87e6-d2af43b35ba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9655f3864c2f42b2b113bf372e92e47c'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:52.715514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-04156da5-fb87-4e96-87e6-d2af43b35ba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9655f3864c2f42b2b113bf372e92e47c'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:52.725085 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-04156da5-fb87-4e96-87e6-d2af43b35ba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9655f3864c2f42b2b113bf372e92e47c'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:52.740561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-04156da5-fb87-4e96-87e6-d2af43b35ba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9655f3864c2f42b2b113bf372e92e47c'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:52.747901 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1144/2287] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:18:52 2022] DELETE /identity/v3/users/9655f3864c2f42b2b113bf372e92e47c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:52.959908 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33f906-8274-4dfa-9c3a-df9e94c48847 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:52.961359 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1c33f906-8274-4dfa-9c3a-df9e94c48847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:52.981600 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33f906-8274-4dfa-9c3a-df9e94c48847 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:53.063988 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa5e6b-168a-4d56-b88c-b4c9e60950c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:53.064719 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa5e6b-168a-4d56-b88c-b4c9e60950c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:53.065004 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa5e6b-168a-4d56-b88c-b4c9e60950c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:53.065276 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa5e6b-168a-4d56-b88c-b4c9e60950c8 admin admin] PATH_INFO: `/v3/projects/674e3602e0b3411cb7db1130b93041cf` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:53.066587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa5e6b-168a-4d56-b88c-b4c9e60950c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=674e3602e0b3411cb7db1130b93041cf)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:53.068943 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa5e6b-168a-4d56-b88c-b4c9e60950c8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:53.131124 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-27aa5e6b-168a-4d56-b88c-b4c9e60950c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '674e3602e0b3411cb7db1130b93041cf', 'request_id': 'req-27aa5e6b-168a-4d56-b88c-b4c9e60950c8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:53.139812 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-27aa5e6b-168a-4d56-b88c-b4c9e60950c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '674e3602e0b3411cb7db1130b93041cf', 'request_id': 'req-27aa5e6b-168a-4d56-b88c-b4c9e60950c8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:53.146410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-27aa5e6b-168a-4d56-b88c-b4c9e60950c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '674e3602e0b3411cb7db1130b93041cf', 'request_id': 'req-27aa5e6b-168a-4d56-b88c-b4c9e60950c8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:53.189984 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-27aa5e6b-168a-4d56-b88c-b4c9e60950c8 admin admin] The token cache is being invalidate because project 674e3602e0b3411cb7db1130b93041cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:18:53.190277 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-27aa5e6b-168a-4d56-b88c-b4c9e60950c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:53.190939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-27aa5e6b-168a-4d56-b88c-b4c9e60950c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:53.192441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1144/2288] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:18:52 2022] DELETE /identity/v3/projects/674e3602e0b3411cb7db1130b93041cf => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:53.197294 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4568e6-0ae3-44f8-83e8-07769a74bcf9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:53.197657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4568e6-0ae3-44f8-83e8-07769a74bcf9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:53.198091 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4568e6-0ae3-44f8-83e8-07769a74bcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:53.222049 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6f4568e6-0ae3-44f8-83e8-07769a74bcf9 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:53.223172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6f4568e6-0ae3-44f8-83e8-07769a74bcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:53.322526 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1145/2289] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:53.323188 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3b4ee-2908-456c-ba6e-eb1ccaf92ced None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:53.323482 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3b4ee-2908-456c-ba6e-eb1ccaf92ced None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:53.323762 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3b4ee-2908-456c-ba6e-eb1ccaf92ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:53.335035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:53.348315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-dfd3b4ee-2908-456c-ba6e-eb1ccaf92ced None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:53.350150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-dfd3b4ee-2908-456c-ba6e-eb1ccaf92ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:53.367479 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4568e6-0ae3-44f8-83e8-07769a74bcf9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:53.367980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1145/2290] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:18:53.378013 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4568e6-0ae3-44f8-83e8-07769a74bcf9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:53.383299 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb912b-075a-4ede-9aba-09872cd7f7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:53.384172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb912b-075a-4ede-9aba-09872cd7f7fb admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:53.384452 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb912b-075a-4ede-9aba-09872cd7f7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:53.384720 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb912b-075a-4ede-9aba-09872cd7f7fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:53.385405 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb912b-075a-4ede-9aba-09872cd7f7fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:18:53.385756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb912b-075a-4ede-9aba-09872cd7f7fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:53.387459 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb912b-075a-4ede-9aba-09872cd7f7fb admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:53.402835 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1146/2291] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:18:53.407112 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3b4ee-2908-456c-ba6e-eb1ccaf92ced None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:53.419098 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3b4ee-2908-456c-ba6e-eb1ccaf92ced None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:53.424214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108dc6b-17bf-4fd4-9949-4ab3456bb522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:53.424828 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108dc6b-17bf-4fd4-9949-4ab3456bb522 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:53.425109 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108dc6b-17bf-4fd4-9949-4ab3456bb522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:53.425381 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108dc6b-17bf-4fd4-9949-4ab3456bb522 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:53.426023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108dc6b-17bf-4fd4-9949-4ab3456bb522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:53.427966 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108dc6b-17bf-4fd4-9949-4ab3456bb522 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:53.463777 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1146/2292] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:18:53 2022] POST /identity/v3/projects => generated 359 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:53.468013 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb912b-075a-4ede-9aba-09872cd7f7fb admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:53.479847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb912b-075a-4ede-9aba-09872cd7f7fb admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:53.507651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:53.568640 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be92a4-0499-470b-93f6-ba7af2975b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:53.569262 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be92a4-0499-470b-93f6-ba7af2975b2a admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:53.569539 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be92a4-0499-470b-93f6-ba7af2975b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:53.569808 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be92a4-0499-470b-93f6-ba7af2975b2a admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:53.570474 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be92a4-0499-470b-93f6-ba7af2975b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:53.574099 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be92a4-0499-470b-93f6-ba7af2975b2a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:53.598377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1147/2293] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:18:53 2022] POST /identity/v3/users => generated 280 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:53.602429 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108dc6b-17bf-4fd4-9949-4ab3456bb522 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:53.616956 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108dc6b-17bf-4fd4-9949-4ab3456bb522 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:53.622473 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c28b8-c353-4e0d-8203-2a7b707b3260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:53.623132 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c28b8-c353-4e0d-8203-2a7b707b3260 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:53.623401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c28b8-c353-4e0d-8203-2a7b707b3260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:53.623775 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c28b8-c353-4e0d-8203-2a7b707b3260 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:53.624400 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c28b8-c353-4e0d-8203-2a7b707b3260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:53.626328 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c28b8-c353-4e0d-8203-2a7b707b3260 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:53.638676 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1147/2294] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:53.642776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be92a4-0499-470b-93f6-ba7af2975b2a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:53.654316 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be92a4-0499-470b-93f6-ba7af2975b2a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:53.659415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58780fa6-6ae8-4377-a7e3-b790262e3738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:53.660031 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58780fa6-6ae8-4377-a7e3-b790262e3738 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:53.660306 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58780fa6-6ae8-4377-a7e3-b790262e3738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:53.660577 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58780fa6-6ae8-4377-a7e3-b790262e3738 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:53.661219 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58780fa6-6ae8-4377-a7e3-b790262e3738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:53.663156 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58780fa6-6ae8-4377-a7e3-b790262e3738 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:53.676970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1148/2295] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:18:53.681165 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c28b8-c353-4e0d-8203-2a7b707b3260 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:53.694274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c28b8-c353-4e0d-8203-2a7b707b3260 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:53.699431 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc33677-d740-44f4-ba37-edc7a1ddeb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:53.700107 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc33677-d740-44f4-ba37-edc7a1ddeb6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:53.700389 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc33677-d740-44f4-ba37-edc7a1ddeb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:53.700736 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc33677-d740-44f4-ba37-edc7a1ddeb6c admin admin] PATH_INFO: `/v3/projects/efb543d9eeca4facb8de185d45b705db/users/a6119a2782cc46c8827cd3d45d5afb8a/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:53.717569 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc33677-d740-44f4-ba37-edc7a1ddeb6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=efb543d9eeca4facb8de185d45b705db, user_id=a6119a2782cc46c8827cd3d45d5afb8a, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:53.720332 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc33677-d740-44f4-ba37-edc7a1ddeb6c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:53.738659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1148/2296] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:53 2022] PUT /identity/v3/projects/efb543d9eeca4facb8de185d45b705db/users/a6119a2782cc46c8827cd3d45d5afb8a/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:53.749232 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1c08a-c9f1-4886-84d7-20af00c9303b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:53.749743 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1c08a-c9f1-4886-84d7-20af00c9303b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:53.749845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1c08a-c9f1-4886-84d7-20af00c9303b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:53.786086 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-01a1c08a-c9f1-4886-84d7-20af00c9303b None None] MFA Rules not processed for user `a6119a2782cc46c8827cd3d45d5afb8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:53.787590 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-01a1c08a-c9f1-4886-84d7-20af00c9303b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:53.867445 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1149/2297] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:53 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:53.907228 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc33677-d740-44f4-ba37-edc7a1ddeb6c admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:53.909235 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8fc33677-d740-44f4-ba37-edc7a1ddeb6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:53.987769 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc33677-d740-44f4-ba37-edc7a1ddeb6c admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:53.991546 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7b2e3-1a4d-4fb6-81a0-0f1c257a4d06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:53.992188 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7b2e3-1a4d-4fb6-81a0-0f1c257a4d06 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:53.993236 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7b2e3-1a4d-4fb6-81a0-0f1c257a4d06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:53.993725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7b2e3-1a4d-4fb6-81a0-0f1c257a4d06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:54.013273 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7b2e3-1a4d-4fb6-81a0-0f1c257a4d06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:54.014788 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7b2e3-1a4d-4fb6-81a0-0f1c257a4d06 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:54.028774 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1149/2298] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:53 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:54.049049 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1c08a-c9f1-4886-84d7-20af00c9303b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:54.128960 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1c08a-c9f1-4886-84d7-20af00c9303b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:54.134170 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e08a6-1943-456e-914a-00b5cdc7520c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:54.135488 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e08a6-1943-456e-914a-00b5cdc7520c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:54.135898 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e08a6-1943-456e-914a-00b5cdc7520c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:54.136298 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e08a6-1943-456e-914a-00b5cdc7520c admin admin] PATH_INFO: `/v3/users/a6119a2782cc46c8827cd3d45d5afb8a` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:54.139689 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e08a6-1943-456e-914a-00b5cdc7520c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6119a2782cc46c8827cd3d45d5afb8a)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:54.142420 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e08a6-1943-456e-914a-00b5cdc7520c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:54.188750 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:54.201941 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ff0e08a6-1943-456e-914a-00b5cdc7520c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6119a2782cc46c8827cd3d45d5afb8a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:54.214634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ff0e08a6-1943-456e-914a-00b5cdc7520c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6119a2782cc46c8827cd3d45d5afb8a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:54.217280 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ff0e08a6-1943-456e-914a-00b5cdc7520c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6119a2782cc46c8827cd3d45d5afb8a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:54.229910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ff0e08a6-1943-456e-914a-00b5cdc7520c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6119a2782cc46c8827cd3d45d5afb8a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:54.246326 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-ff0e08a6-1943-456e-914a-00b5cdc7520c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6119a2782cc46c8827cd3d45d5afb8a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:54.253214 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1150/2299] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:18:54 2022] DELETE /identity/v3/users/a6119a2782cc46c8827cd3d45d5afb8a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:18:54.257944 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1d490-9ccd-4630-8768-c20d8d8a0ce5 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:54.258420 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1d490-9ccd-4630-8768-c20d8d8a0ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:54.258758 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1d490-9ccd-4630-8768-c20d8d8a0ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:54.303867 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-21e1d490-9ccd-4630-8768-c20d8d8a0ce5 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:54.305393 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-21e1d490-9ccd-4630-8768-c20d8d8a0ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:54.410845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1150/2300] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:54.419424 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e08a6-1943-456e-914a-00b5cdc7520c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:54.422778 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ff0e08a6-1943-456e-914a-00b5cdc7520c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:54.512648 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e08a6-1943-456e-914a-00b5cdc7520c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:54.517247 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e2962-504c-47a9-a22a-1d1cfffd2b66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:54.518138 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e2962-504c-47a9-a22a-1d1cfffd2b66 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:54.518589 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e2962-504c-47a9-a22a-1d1cfffd2b66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:54.519043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e2962-504c-47a9-a22a-1d1cfffd2b66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:54.531634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e2962-504c-47a9-a22a-1d1cfffd2b66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:54.533118 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e2962-504c-47a9-a22a-1d1cfffd2b66 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:54.557924 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1151/2301] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:54.988211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1d490-9ccd-4630-8768-c20d8d8a0ce5 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:54.989666 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-21e1d490-9ccd-4630-8768-c20d8d8a0ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:55.012845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1d490-9ccd-4630-8768-c20d8d8a0ce5 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:55.020150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b4ce2-10bd-4faa-b7b5-0d366f9e2973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:55.020808 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b4ce2-10bd-4faa-b7b5-0d366f9e2973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:55.021089 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b4ce2-10bd-4faa-b7b5-0d366f9e2973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:55.021362 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b4ce2-10bd-4faa-b7b5-0d366f9e2973 admin admin] PATH_INFO: `/v3/projects/efb543d9eeca4facb8de185d45b705db` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:55.022533 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b4ce2-10bd-4faa-b7b5-0d366f9e2973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efb543d9eeca4facb8de185d45b705db)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:55.024448 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b4ce2-10bd-4faa-b7b5-0d366f9e2973 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:55.087810 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:55.102124 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fc5b4ce2-10bd-4faa-b7b5-0d366f9e2973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efb543d9eeca4facb8de185d45b705db', 'request_id': 'req-fc5b4ce2-10bd-4faa-b7b5-0d366f9e2973'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:55.110213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fc5b4ce2-10bd-4faa-b7b5-0d366f9e2973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efb543d9eeca4facb8de185d45b705db', 'request_id': 'req-fc5b4ce2-10bd-4faa-b7b5-0d366f9e2973'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:55.117717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fc5b4ce2-10bd-4faa-b7b5-0d366f9e2973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efb543d9eeca4facb8de185d45b705db', 'request_id': 'req-fc5b4ce2-10bd-4faa-b7b5-0d366f9e2973'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:55.135327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fc5b4ce2-10bd-4faa-b7b5-0d366f9e2973 admin admin] The token cache is being invalidate because project efb543d9eeca4facb8de185d45b705db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:18:55.135906 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fc5b4ce2-10bd-4faa-b7b5-0d366f9e2973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:55.136768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-fc5b4ce2-10bd-4faa-b7b5-0d366f9e2973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:55.138279 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1151/2302] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:18:54 2022] DELETE /identity/v3/projects/efb543d9eeca4facb8de185d45b705db => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:55.143323 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c4595-db1d-47dd-b48d-0972a3efc91c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:55.145910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c4595-db1d-47dd-b48d-0972a3efc91c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:55.145910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c4595-db1d-47dd-b48d-0972a3efc91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:55.167073 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f93c4595-db1d-47dd-b48d-0972a3efc91c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:55.168796 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f93c4595-db1d-47dd-b48d-0972a3efc91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:55.241239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1152/2303] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:55.253295 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a3589-320d-4904-b56b-432872383cf9 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:55.253795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a3589-320d-4904-b56b-432872383cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:55.254090 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a3589-320d-4904-b56b-432872383cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:55.286134 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-491a3589-320d-4904-b56b-432872383cf9 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:55.288751 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-491a3589-320d-4904-b56b-432872383cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:55.309191 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1152/2304] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:55.310385 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c4595-db1d-47dd-b48d-0972a3efc91c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:55.334274 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c4595-db1d-47dd-b48d-0972a3efc91c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:55.345329 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9f31b-19b2-472e-afe4-a918380121b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:55.346014 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9f31b-19b2-472e-afe4-a918380121b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:55.346297 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9f31b-19b2-472e-afe4-a918380121b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:55.346572 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9f31b-19b2-472e-afe4-a918380121b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:55.347266 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9f31b-19b2-472e-afe4-a918380121b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:18:55.347572 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9f31b-19b2-472e-afe4-a918380121b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:55.349178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9f31b-19b2-472e-afe4-a918380121b2 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:55.377959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1153/2305] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:18:55.379595 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a3589-320d-4904-b56b-432872383cf9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:55.397690 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a3589-320d-4904-b56b-432872383cf9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:55.414112 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377b8e1-08db-4b6d-adb8-b374d63d1319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:55.415158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377b8e1-08db-4b6d-adb8-b374d63d1319 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:55.415443 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377b8e1-08db-4b6d-adb8-b374d63d1319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:55.415715 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7377b8e1-08db-4b6d-adb8-b374d63d1319 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:55.416896 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7377b8e1-08db-4b6d-adb8-b374d63d1319 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:55.419075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7377b8e1-08db-4b6d-adb8-b374d63d1319 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:55.452801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1153/2306] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:18:55 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:18:55.456935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9f31b-19b2-472e-afe4-a918380121b2 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:55.469716 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9f31b-19b2-472e-afe4-a918380121b2 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:55.518983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:55.547976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024d3f1-1243-491e-b3c1-de8239c86bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:55.548628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024d3f1-1243-491e-b3c1-de8239c86bac admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:55.548908 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024d3f1-1243-491e-b3c1-de8239c86bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:55.549181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024d3f1-1243-491e-b3c1-de8239c86bac admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:55.549832 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024d3f1-1243-491e-b3c1-de8239c86bac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:55.551864 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024d3f1-1243-491e-b3c1-de8239c86bac admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:55.581741 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1154/2307] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:18:55 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:55.583551 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377b8e1-08db-4b6d-adb8-b374d63d1319 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:55.594524 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7377b8e1-08db-4b6d-adb8-b374d63d1319 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:55.599211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b5016-00a8-4f66-bc76-c9c0c22bc580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:55.599814 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b5016-00a8-4f66-bc76-c9c0c22bc580 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:55.600090 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b5016-00a8-4f66-bc76-c9c0c22bc580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:55.600358 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b5016-00a8-4f66-bc76-c9c0c22bc580 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:55.601000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b5016-00a8-4f66-bc76-c9c0c22bc580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:55.605491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b5016-00a8-4f66-bc76-c9c0c22bc580 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:55.616730 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1154/2308] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:55.621352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024d3f1-1243-491e-b3c1-de8239c86bac admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:55.633515 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024d3f1-1243-491e-b3c1-de8239c86bac admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:55.638799 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2678ad-8274-4a36-bb84-7383251b3795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:55.639463 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2678ad-8274-4a36-bb84-7383251b3795 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:55.639725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2678ad-8274-4a36-bb84-7383251b3795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:55.640015 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2678ad-8274-4a36-bb84-7383251b3795 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:55.641231 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2678ad-8274-4a36-bb84-7383251b3795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:55.642728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2678ad-8274-4a36-bb84-7383251b3795 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:55.655472 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1155/2309] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:18:55.659660 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b5016-00a8-4f66-bc76-c9c0c22bc580 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:55.673247 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b5016-00a8-4f66-bc76-c9c0c22bc580 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:55.678054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19361355-9823-48a3-b30d-ed16ef29dff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:55.678745 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19361355-9823-48a3-b30d-ed16ef29dff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:55.679035 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19361355-9823-48a3-b30d-ed16ef29dff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:55.679315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19361355-9823-48a3-b30d-ed16ef29dff7 admin admin] PATH_INFO: `/v3/projects/a3740bdc97e241bba60d94020ff51748/users/6e825afc2d614c338b37dbb7f8a04ec3/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:55.700475 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19361355-9823-48a3-b30d-ed16ef29dff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3740bdc97e241bba60d94020ff51748, user_id=6e825afc2d614c338b37dbb7f8a04ec3, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:55.703403 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19361355-9823-48a3-b30d-ed16ef29dff7 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:55.720477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1155/2310] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:55 2022] PUT /identity/v3/projects/a3740bdc97e241bba60d94020ff51748/users/6e825afc2d614c338b37dbb7f8a04ec3/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:18:55.732056 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc58d18-c747-4dcd-b1b6-26bc6ae62faf None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:55.732378 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc58d18-c747-4dcd-b1b6-26bc6ae62faf None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:55.732623 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc58d18-c747-4dcd-b1b6-26bc6ae62faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:55.775412 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-5bc58d18-c747-4dcd-b1b6-26bc6ae62faf None None] MFA Rules not processed for user `6e825afc2d614c338b37dbb7f8a04ec3`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:55.775699 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5bc58d18-c747-4dcd-b1b6-26bc6ae62faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:55.903838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1156/2311] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:55.904910 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19361355-9823-48a3-b30d-ed16ef29dff7 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:56.062845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19361355-9823-48a3-b30d-ed16ef29dff7 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:56.071231 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951553d-e2e8-4fcb-8029-6be5c22575c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:56.072406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951553d-e2e8-4fcb-8029-6be5c22575c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:56.073624 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951553d-e2e8-4fcb-8029-6be5c22575c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:56.073911 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951553d-e2e8-4fcb-8029-6be5c22575c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:56.077522 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951553d-e2e8-4fcb-8029-6be5c22575c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:56.079745 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951553d-e2e8-4fcb-8029-6be5c22575c5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:56.114969 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1156/2312] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:18:55 2022] POST /identity/v3/projects => generated 373 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:56.123944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc58d18-c747-4dcd-b1b6-26bc6ae62faf None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:56.160077 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc58d18-c747-4dcd-b1b6-26bc6ae62faf None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:56.309860 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35fa327-66db-4c0a-bc78-12bb354959d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:56.310513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fa327-66db-4c0a-bc78-12bb354959d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:56.310794 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fa327-66db-4c0a-bc78-12bb354959d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:56.311109 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fa327-66db-4c0a-bc78-12bb354959d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:56.311934 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35fa327-66db-4c0a-bc78-12bb354959d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:56.313895 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35fa327-66db-4c0a-bc78-12bb354959d3 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:56.340845 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1157/2313] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:18:56 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:56.344971 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951553d-e2e8-4fcb-8029-6be5c22575c5 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:56.356534 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951553d-e2e8-4fcb-8029-6be5c22575c5 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:56.362993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbd3ab-4b33-468a-a659-e065aa4213cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:56.363681 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbd3ab-4b33-468a-a659-e065aa4213cc admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:56.364030 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbd3ab-4b33-468a-a659-e065aa4213cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:56.364363 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbd3ab-4b33-468a-a659-e065aa4213cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:56.365073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbd3ab-4b33-468a-a659-e065aa4213cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:56.366928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbd3ab-4b33-468a-a659-e065aa4213cc admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:56.380538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1157/2314] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:56.385453 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35fa327-66db-4c0a-bc78-12bb354959d3 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:56.396596 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35fa327-66db-4c0a-bc78-12bb354959d3 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:56.401506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027f4ea-c41e-4da8-aa74-63c97c07f7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:56.402175 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027f4ea-c41e-4da8-aa74-63c97c07f7a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:56.405606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027f4ea-c41e-4da8-aa74-63c97c07f7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:56.405606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027f4ea-c41e-4da8-aa74-63c97c07f7a7 admin admin] PATH_INFO: `/v3/projects/72b6366bd24846a297f5e4174cfe4eb4/users/64a3290ba445439389d03f8803112c9f/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:56.426601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027f4ea-c41e-4da8-aa74-63c97c07f7a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72b6366bd24846a297f5e4174cfe4eb4, user_id=64a3290ba445439389d03f8803112c9f, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:56.429088 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027f4ea-c41e-4da8-aa74-63c97c07f7a7 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:56.444605 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1158/2315] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:56 2022] PUT /identity/v3/projects/72b6366bd24846a297f5e4174cfe4eb4/users/64a3290ba445439389d03f8803112c9f/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:56.448851 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbd3ab-4b33-468a-a659-e065aa4213cc admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:56.459912 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbd3ab-4b33-468a-a659-e065aa4213cc admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:56.502776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:56.533073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac46660-2a34-47d7-adde-a578a9c47cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:56.534078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac46660-2a34-47d7-adde-a578a9c47cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:56.534638 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac46660-2a34-47d7-adde-a578a9c47cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:56.534942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac46660-2a34-47d7-adde-a578a9c47cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:56.535649 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac46660-2a34-47d7-adde-a578a9c47cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:56.537655 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac46660-2a34-47d7-adde-a578a9c47cc9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:56.554504 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1158/2316] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:56.556287 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027f4ea-c41e-4da8-aa74-63c97c07f7a7 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:56.568385 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027f4ea-c41e-4da8-aa74-63c97c07f7a7 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:56.574120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb001e57-0a73-433d-9f12-eaa042690a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:56.574798 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb001e57-0a73-433d-9f12-eaa042690a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:56.575108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb001e57-0a73-433d-9f12-eaa042690a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:56.575380 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb001e57-0a73-433d-9f12-eaa042690a88 admin admin] PATH_INFO: `/v3/projects/72b6366bd24846a297f5e4174cfe4eb4/users/64a3290ba445439389d03f8803112c9f/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:56.577434 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb001e57-0a73-433d-9f12-eaa042690a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72b6366bd24846a297f5e4174cfe4eb4, user_id=64a3290ba445439389d03f8803112c9f, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:56.580272 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb001e57-0a73-433d-9f12-eaa042690a88 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:56.592429 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1159/2317] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:56 2022] PUT /identity/v3/projects/72b6366bd24846a297f5e4174cfe4eb4/users/64a3290ba445439389d03f8803112c9f/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:56.603283 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dce588-9972-4bed-9c93-5dd94b307551 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:56.603587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dce588-9972-4bed-9c93-5dd94b307551 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:56.604440 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dce588-9972-4bed-9c93-5dd94b307551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:56.636586 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-22dce588-9972-4bed-9c93-5dd94b307551 None None] MFA Rules not processed for user `64a3290ba445439389d03f8803112c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:56.638015 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-22dce588-9972-4bed-9c93-5dd94b307551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:56.719278 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1159/2318] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:56.730636 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb001e57-0a73-433d-9f12-eaa042690a88 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:56.732468 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-eb001e57-0a73-433d-9f12-eaa042690a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:56.737908 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:56.797983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb001e57-0a73-433d-9f12-eaa042690a88 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:56.801755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce96b6-4004-4b25-ab15-9694b5170642 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:56.802362 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce96b6-4004-4b25-ab15-9694b5170642 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:56.802632 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce96b6-4004-4b25-ab15-9694b5170642 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:56.803026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce96b6-4004-4b25-ab15-9694b5170642 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:56.865842 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce96b6-4004-4b25-ab15-9694b5170642 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:56.867343 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edce96b6-4004-4b25-ab15-9694b5170642 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:56.879301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1160/2319] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:56.935792 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de41da-e5de-4ece-958b-3e7c9d3803e9 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:56.936086 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de41da-e5de-4ece-958b-3e7c9d3803e9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:56.936369 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de41da-e5de-4ece-958b-3e7c9d3803e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:56.955922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a9de41da-e5de-4ece-958b-3e7c9d3803e9 None None] MFA Rules not processed for user `82da4b8f8bdb4dbab655ff510bfa328f`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:56.957105 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a9de41da-e5de-4ece-958b-3e7c9d3803e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:57.014471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1160/2320] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:57.049349 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce96b6-4004-4b25-ab15-9694b5170642 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:57.050896 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-edce96b6-4004-4b25-ab15-9694b5170642 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:57.139568 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce96b6-4004-4b25-ab15-9694b5170642 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:57.158126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c9457-4d6f-4fdb-b86d-20ddc26b7806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:57.159224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c9457-4d6f-4fdb-b86d-20ddc26b7806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:57.159580 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c9457-4d6f-4fdb-b86d-20ddc26b7806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:57.159945 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c9457-4d6f-4fdb-b86d-20ddc26b7806 admin admin] PATH_INFO: `/v3/users/dcda6d1c205d4ec0bd4adcd65fa25248` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:57.162546 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c9457-4d6f-4fdb-b86d-20ddc26b7806 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcda6d1c205d4ec0bd4adcd65fa25248)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:57.164256 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c9457-4d6f-4fdb-b86d-20ddc26b7806 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:57.232661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-178c9457-4d6f-4fdb-b86d-20ddc26b7806 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcda6d1c205d4ec0bd4adcd65fa25248'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:57.242843 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-178c9457-4d6f-4fdb-b86d-20ddc26b7806 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcda6d1c205d4ec0bd4adcd65fa25248'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:57.243624 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-178c9457-4d6f-4fdb-b86d-20ddc26b7806 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcda6d1c205d4ec0bd4adcd65fa25248'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:57.248023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:57.255056 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-178c9457-4d6f-4fdb-b86d-20ddc26b7806 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcda6d1c205d4ec0bd4adcd65fa25248'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:57.281941 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-178c9457-4d6f-4fdb-b86d-20ddc26b7806 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcda6d1c205d4ec0bd4adcd65fa25248'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:57.289187 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1161/2321] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:18:57 2022] DELETE /identity/v3/users/dcda6d1c205d4ec0bd4adcd65fa25248 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:57.293815 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3979b-717f-4f0f-8149-775a9cfc86b7 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:57.294107 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3979b-717f-4f0f-8149-775a9cfc86b7 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:57.294384 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3979b-717f-4f0f-8149-775a9cfc86b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:57.317839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b6f3979b-717f-4f0f-8149-775a9cfc86b7 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:57.319131 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b6f3979b-717f-4f0f-8149-775a9cfc86b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:57.336430 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c9457-4d6f-4fdb-b86d-20ddc26b7806 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:57.339958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-178c9457-4d6f-4fdb-b86d-20ddc26b7806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:57.388541 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1161/2322] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:18:57.395094 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3979b-717f-4f0f-8149-775a9cfc86b7 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:57.397496 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b6f3979b-717f-4f0f-8149-775a9cfc86b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:57.405401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:57.415702 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c9457-4d6f-4fdb-b86d-20ddc26b7806 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:57.416844 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201c4f6-d357-4eb3-93c7-d1683d58b9e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:57.417473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201c4f6-d357-4eb3-93c7-d1683d58b9e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:57.417767 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201c4f6-d357-4eb3-93c7-d1683d58b9e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:57.418021 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201c4f6-d357-4eb3-93c7-d1683d58b9e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:57.419830 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a201c4f6-d357-4eb3-93c7-d1683d58b9e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:57.430106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201c4f6-d357-4eb3-93c7-d1683d58b9e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:57.431572 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201c4f6-d357-4eb3-93c7-d1683d58b9e7 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:57.443049 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1162/2323] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:18:57.467367 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3979b-717f-4f0f-8149-775a9cfc86b7 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:57.471223 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94cd71-3caf-4124-9321-a3b7e5c02cf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:57.471839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94cd71-3caf-4124-9321-a3b7e5c02cf9 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:57.472212 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94cd71-3caf-4124-9321-a3b7e5c02cf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:57.472372 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94cd71-3caf-4124-9321-a3b7e5c02cf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:57.482312 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94cd71-3caf-4124-9321-a3b7e5c02cf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:57.483782 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94cd71-3caf-4124-9321-a3b7e5c02cf9 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:57.495584 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1162/2324] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:57.798404 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201c4f6-d357-4eb3-93c7-d1683d58b9e7 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:57.799891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a201c4f6-d357-4eb3-93c7-d1683d58b9e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:57.814641 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201c4f6-d357-4eb3-93c7-d1683d58b9e7 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:57.819329 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46f721-e23f-4c9c-8a3e-8bcbf9b2e9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:57.819970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46f721-e23f-4c9c-8a3e-8bcbf9b2e9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:57.820250 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46f721-e23f-4c9c-8a3e-8bcbf9b2e9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:57.820524 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46f721-e23f-4c9c-8a3e-8bcbf9b2e9e8 admin admin] PATH_INFO: `/v3/projects/dc478db2bb82408194391f5e0165d7b0` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:57.821672 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46f721-e23f-4c9c-8a3e-8bcbf9b2e9e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc478db2bb82408194391f5e0165d7b0)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:57.824257 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46f721-e23f-4c9c-8a3e-8bcbf9b2e9e8 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:57.901856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-8b46f721-e23f-4c9c-8a3e-8bcbf9b2e9e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc478db2bb82408194391f5e0165d7b0', 'request_id': 'req-8b46f721-e23f-4c9c-8a3e-8bcbf9b2e9e8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:57.907431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-8b46f721-e23f-4c9c-8a3e-8bcbf9b2e9e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc478db2bb82408194391f5e0165d7b0', 'request_id': 'req-8b46f721-e23f-4c9c-8a3e-8bcbf9b2e9e8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:57.914206 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-8b46f721-e23f-4c9c-8a3e-8bcbf9b2e9e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc478db2bb82408194391f5e0165d7b0', 'request_id': 'req-8b46f721-e23f-4c9c-8a3e-8bcbf9b2e9e8'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:57.928398 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-8b46f721-e23f-4c9c-8a3e-8bcbf9b2e9e8 admin admin] The token cache is being invalidate because project dc478db2bb82408194391f5e0165d7b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:18:57.928725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-8b46f721-e23f-4c9c-8a3e-8bcbf9b2e9e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:57.929360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-8b46f721-e23f-4c9c-8a3e-8bcbf9b2e9e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:57.930997 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1163/2325] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:18:57 2022] DELETE /identity/v3/projects/dc478db2bb82408194391f5e0165d7b0 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:57.935817 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e7bac-7558-450c-a408-095695789f49 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:57.936097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e7bac-7558-450c-a408-095695789f49 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:57.936370 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e7bac-7558-450c-a408-095695789f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:57.957258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-2e3e7bac-7558-450c-a408-095695789f49 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:57.958454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2e3e7bac-7558-450c-a408-095695789f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:58.024214 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1163/2326] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:58.033065 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50acba-603a-4c0e-9492-0ace85e296ff None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:58.033220 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50acba-603a-4c0e-9492-0ace85e296ff None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:58.033840 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50acba-603a-4c0e-9492-0ace85e296ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:58.054705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-7b50acba-603a-4c0e-9492-0ace85e296ff None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:58.055959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7b50acba-603a-4c0e-9492-0ace85e296ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:58.064693 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1164/2327] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:58.068996 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e7bac-7558-450c-a408-095695789f49 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:58.079857 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e7bac-7558-450c-a408-095695789f49 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:58.084840 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1928b8-2e95-40d4-aec4-933937d8a1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:58.085447 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1928b8-2e95-40d4-aec4-933937d8a1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:58.085727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1928b8-2e95-40d4-aec4-933937d8a1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:58.085999 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1928b8-2e95-40d4-aec4-933937d8a1b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:58.086747 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1928b8-2e95-40d4-aec4-933937d8a1b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:18:58.087158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1928b8-2e95-40d4-aec4-933937d8a1b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:58.089988 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1928b8-2e95-40d4-aec4-933937d8a1b8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:58.104863 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1164/2328] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:18:58.109357 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50acba-603a-4c0e-9492-0ace85e296ff None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:58.118669 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50acba-603a-4c0e-9492-0ace85e296ff None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:58.123725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d900075-6051-4b54-885e-1c762c78efae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:58.124332 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d900075-6051-4b54-885e-1c762c78efae admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:58.124618 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d900075-6051-4b54-885e-1c762c78efae admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:58.124891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d900075-6051-4b54-885e-1c762c78efae admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:58.125540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d900075-6051-4b54-885e-1c762c78efae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:58.127483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d900075-6051-4b54-885e-1c762c78efae admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:58.160172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1165/2329] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:18:58 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:58.161334 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1928b8-2e95-40d4-aec4-933937d8a1b8 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:58.172422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1928b8-2e95-40d4-aec4-933937d8a1b8 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:58.232780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb523f-c94e-4c10-880b-8da1a910a841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:58.233395 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb523f-c94e-4c10-880b-8da1a910a841 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:58.233675 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb523f-c94e-4c10-880b-8da1a910a841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:58.233995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb523f-c94e-4c10-880b-8da1a910a841 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:58.234631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb523f-c94e-4c10-880b-8da1a910a841 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:58.236747 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb523f-c94e-4c10-880b-8da1a910a841 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:58.263420 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1165/2330] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:18:58 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:18:58.264890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d900075-6051-4b54-885e-1c762c78efae admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:58.284912 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d900075-6051-4b54-885e-1c762c78efae admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:58.290805 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353ce50-0a89-4711-a84f-6f279cd870f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:58.292060 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353ce50-0a89-4711-a84f-6f279cd870f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:58.292344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353ce50-0a89-4711-a84f-6f279cd870f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:58.292828 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353ce50-0a89-4711-a84f-6f279cd870f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:58.293951 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353ce50-0a89-4711-a84f-6f279cd870f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:58.295611 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353ce50-0a89-4711-a84f-6f279cd870f4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:58.306613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1166/2331] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:18:58.310753 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb523f-c94e-4c10-880b-8da1a910a841 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:58.321658 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb523f-c94e-4c10-880b-8da1a910a841 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:58.326584 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f978d5-147a-4c9e-8613-34ae4c82870a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:58.327245 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f978d5-147a-4c9e-8613-34ae4c82870a admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:58.327537 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f978d5-147a-4c9e-8613-34ae4c82870a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:58.327797 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f978d5-147a-4c9e-8613-34ae4c82870a admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:58.328481 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f978d5-147a-4c9e-8613-34ae4c82870a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:58.330096 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f978d5-147a-4c9e-8613-34ae4c82870a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:58.343165 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1166/2332] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:18:58.347209 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353ce50-0a89-4711-a84f-6f279cd870f4 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:58.357966 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353ce50-0a89-4711-a84f-6f279cd870f4 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:58.363355 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a3071-916b-4ede-8650-b3d8752fcd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:58.364023 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a3071-916b-4ede-8650-b3d8752fcd3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:58.364504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a3071-916b-4ede-8650-b3d8752fcd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:58.364728 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a3071-916b-4ede-8650-b3d8752fcd3c admin admin] PATH_INFO: `/v3/projects/6d623783bda1452aa94058ca8ea1e2b3/users/3862054580664f8494c5583ee8334ecb/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:58.380133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a3071-916b-4ede-8650-b3d8752fcd3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d623783bda1452aa94058ca8ea1e2b3, user_id=3862054580664f8494c5583ee8334ecb, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:58.383314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85a3071-916b-4ede-8650-b3d8752fcd3c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:58.392251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:58.399170 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1167/2333] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:58 2022] PUT /identity/v3/projects/6d623783bda1452aa94058ca8ea1e2b3/users/3862054580664f8494c5583ee8334ecb/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:58.411149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe08ea3-839f-4038-b78c-4ef237ad580c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:58.411450 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe08ea3-839f-4038-b78c-4ef237ad580c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:58.411967 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe08ea3-839f-4038-b78c-4ef237ad580c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:58.445185 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:58.450337 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-dfe08ea3-839f-4038-b78c-4ef237ad580c None None] MFA Rules not processed for user `3862054580664f8494c5583ee8334ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:58.452430 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-dfe08ea3-839f-4038-b78c-4ef237ad580c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:58.532452 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1167/2334] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:58.537346 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a3071-916b-4ede-8650-b3d8752fcd3c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:58.601929 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a3071-916b-4ede-8650-b3d8752fcd3c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:58.608632 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10761f-3904-436c-acd1-d8970dbee5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:58.608632 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10761f-3904-436c-acd1-d8970dbee5fc admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:58.608632 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10761f-3904-436c-acd1-d8970dbee5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:58.608956 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10761f-3904-436c-acd1-d8970dbee5fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:58.609533 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb10761f-3904-436c-acd1-d8970dbee5fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:58.611474 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb10761f-3904-436c-acd1-d8970dbee5fc admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:58.642907 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1168/2335] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:18:58 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:58.644387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe08ea3-839f-4038-b78c-4ef237ad580c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:58.657791 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe08ea3-839f-4038-b78c-4ef237ad580c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:58.719735 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5013e-3893-4e84-9f49-a0c661a24992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:58.720350 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5013e-3893-4e84-9f49-a0c661a24992 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:58.720631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5013e-3893-4e84-9f49-a0c661a24992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:58.720903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5013e-3893-4e84-9f49-a0c661a24992 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:58.721544 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5013e-3893-4e84-9f49-a0c661a24992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:58.722054 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10761f-3904-436c-acd1-d8970dbee5fc admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:58.723661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5013e-3893-4e84-9f49-a0c661a24992 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:58.723800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cb10761f-3904-436c-acd1-d8970dbee5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:58.736583 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10761f-3904-436c-acd1-d8970dbee5fc admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:58.742008 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e961c5c-cd46-4259-88fc-7322baf7f701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:58.742851 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e961c5c-cd46-4259-88fc-7322baf7f701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:58.743376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e961c5c-cd46-4259-88fc-7322baf7f701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:58.743614 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e961c5c-cd46-4259-88fc-7322baf7f701 admin admin] PATH_INFO: `/v3/users/d60a98fb145f497c8b4976c1522aa96a` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:58.745365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e961c5c-cd46-4259-88fc-7322baf7f701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d60a98fb145f497c8b4976c1522aa96a)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:58.747396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e961c5c-cd46-4259-88fc-7322baf7f701 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:58.750710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1168/2336] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:18:58 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:18:58.751628 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5013e-3893-4e84-9f49-a0c661a24992 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:58.761167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5013e-3893-4e84-9f49-a0c661a24992 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:58.766014 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe2340-8363-40b3-814f-76ed29d88d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:58.766828 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe2340-8363-40b3-814f-76ed29d88d52 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:58.767152 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe2340-8363-40b3-814f-76ed29d88d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:58.767424 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe2340-8363-40b3-814f-76ed29d88d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:58.768112 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe2340-8363-40b3-814f-76ed29d88d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:58.769702 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe2340-8363-40b3-814f-76ed29d88d52 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:58.781706 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1169/2337] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:18:58.785977 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe2340-8363-40b3-814f-76ed29d88d52 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:58.798688 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe2340-8363-40b3-814f-76ed29d88d52 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:58.801039 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2e961c5c-cd46-4259-88fc-7322baf7f701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd60a98fb145f497c8b4976c1522aa96a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:58.803453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04daaf-2d82-4fe1-8e94-9ecb0ee7df77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:58.804119 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04daaf-2d82-4fe1-8e94-9ecb0ee7df77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:58.804400 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04daaf-2d82-4fe1-8e94-9ecb0ee7df77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:58.804678 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04daaf-2d82-4fe1-8e94-9ecb0ee7df77 admin admin] PATH_INFO: `/v3/projects/b2a1f59ad2ea49b2bfe6a2887ccd3494/users/c0074a560a0642e8bf7c0435cec5bb15/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:58.809740 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2e961c5c-cd46-4259-88fc-7322baf7f701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd60a98fb145f497c8b4976c1522aa96a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:58.810540 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2e961c5c-cd46-4259-88fc-7322baf7f701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd60a98fb145f497c8b4976c1522aa96a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:58.818503 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2e961c5c-cd46-4259-88fc-7322baf7f701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd60a98fb145f497c8b4976c1522aa96a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:58.823722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04daaf-2d82-4fe1-8e94-9ecb0ee7df77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a1f59ad2ea49b2bfe6a2887ccd3494, user_id=c0074a560a0642e8bf7c0435cec5bb15, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:58.827113 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff04daaf-2d82-4fe1-8e94-9ecb0ee7df77 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:58.833371 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2e961c5c-cd46-4259-88fc-7322baf7f701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd60a98fb145f497c8b4976c1522aa96a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:58.841767 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1169/2338] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:18:58 2022] DELETE /identity/v3/users/d60a98fb145f497c8b4976c1522aa96a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:58.845085 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1170/2339] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:58 2022] PUT /identity/v3/projects/b2a1f59ad2ea49b2bfe6a2887ccd3494/users/c0074a560a0642e8bf7c0435cec5bb15/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:58.846990 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04daaf-2d82-4fe1-8e94-9ecb0ee7df77 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:58.848221 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-ff04daaf-2d82-4fe1-8e94-9ecb0ee7df77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:58.851425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0689e8d-30f8-4097-a32d-ac35ee62b9a9 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:58.851730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0689e8d-30f8-4097-a32d-ac35ee62b9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:58.851983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0689e8d-30f8-4097-a32d-ac35ee62b9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:58.876880 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-f0689e8d-30f8-4097-a32d-ac35ee62b9a9 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:58.878101 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f0689e8d-30f8-4097-a32d-ac35ee62b9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:58.936617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:58.941126 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04daaf-2d82-4fe1-8e94-9ecb0ee7df77 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:58.948947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:58.967732 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1170/2340] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:58.987173 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0689e8d-30f8-4097-a32d-ac35ee62b9a9 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:58.989353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f0689e8d-30f8-4097-a32d-ac35ee62b9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:59.046899 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb5f90-dd39-4689-bb35-b9f300e0d339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:59.048100 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb5f90-dd39-4689-bb35-b9f300e0d339 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:59.048100 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb5f90-dd39-4689-bb35-b9f300e0d339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:59.048474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb5f90-dd39-4689-bb35-b9f300e0d339 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:59.049466 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb5f90-dd39-4689-bb35-b9f300e0d339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:59.050957 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb5f90-dd39-4689-bb35-b9f300e0d339 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:59.067033 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1171/2341] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:18:59.070919 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb5f90-dd39-4689-bb35-b9f300e0d339 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:59.085499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb5f90-dd39-4689-bb35-b9f300e0d339 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:59.091581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9fdbe5-6fa7-45ea-b9d0-c24c87cc5797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:59.092257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fdbe5-6fa7-45ea-b9d0-c24c87cc5797 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:59.092469 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0689e8d-30f8-4097-a32d-ac35ee62b9a9 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:59.092697 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fdbe5-6fa7-45ea-b9d0-c24c87cc5797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:59.092995 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fdbe5-6fa7-45ea-b9d0-c24c87cc5797 admin admin] PATH_INFO: `/v3/projects/b2a1f59ad2ea49b2bfe6a2887ccd3494/users/c0074a560a0642e8bf7c0435cec5bb15/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:59.094945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9fdbe5-6fa7-45ea-b9d0-c24c87cc5797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a1f59ad2ea49b2bfe6a2887ccd3494, user_id=c0074a560a0642e8bf7c0435cec5bb15, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:59.096774 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e34aa-76e7-4433-b3d0-31a18f12b142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:59.097475 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e34aa-76e7-4433-b3d0-31a18f12b142 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:59.097605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9fdbe5-6fa7-45ea-b9d0-c24c87cc5797 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:59.097876 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e34aa-76e7-4433-b3d0-31a18f12b142 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:59.098213 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e34aa-76e7-4433-b3d0-31a18f12b142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:59.108948 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e34aa-76e7-4433-b3d0-31a18f12b142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:59.110046 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1172/2342] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:18:59 2022] PUT /identity/v3/projects/b2a1f59ad2ea49b2bfe6a2887ccd3494/users/c0074a560a0642e8bf7c0435cec5bb15/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:59.110854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e34aa-76e7-4433-b3d0-31a18f12b142 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:59.121973 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17267791-fe96-4624-88a3-e2440d3c7441 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:59.122257 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17267791-fe96-4624-88a3-e2440d3c7441 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:59.122527 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17267791-fe96-4624-88a3-e2440d3c7441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:59.123342 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1171/2343] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:59.153598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-17267791-fe96-4624-88a3-e2440d3c7441 None None] MFA Rules not processed for user `c0074a560a0642e8bf7c0435cec5bb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:59.154804 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-17267791-fe96-4624-88a3-e2440d3c7441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:59.232220 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1173/2344] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:59.245140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e34aa-76e7-4433-b3d0-31a18f12b142 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:59.246831 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e05e34aa-76e7-4433-b3d0-31a18f12b142 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:59.284259 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328460be-3d42-412e-a745-4cc4e169187f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:59.284657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328460be-3d42-412e-a745-4cc4e169187f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:59.285011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328460be-3d42-412e-a745-4cc4e169187f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:59.309306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-328460be-3d42-412e-a745-4cc4e169187f None None] MFA Rules not processed for user `280d47ed79f14105a83b18b2cac58b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:18:59.310856 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-328460be-3d42-412e-a745-4cc4e169187f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:59.315048 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e34aa-76e7-4433-b3d0-31a18f12b142 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:59.369656 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7913878-3abc-4feb-9e9e-dc3104f8b4e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:59.370409 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7913878-3abc-4feb-9e9e-dc3104f8b4e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:59.370817 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7913878-3abc-4feb-9e9e-dc3104f8b4e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:59.371222 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7913878-3abc-4feb-9e9e-dc3104f8b4e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:59.381715 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7913878-3abc-4feb-9e9e-dc3104f8b4e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:59.383172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7913878-3abc-4feb-9e9e-dc3104f8b4e5 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:59.390532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1174/2345] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Oct 17 03:18:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:18:59.395218 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1172/2346] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:18:59.407877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328460be-3d42-412e-a745-4cc4e169187f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:59.409635 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-328460be-3d42-412e-a745-4cc4e169187f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:59.442038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:59.485511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7913878-3abc-4feb-9e9e-dc3104f8b4e5 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:59.496265 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7913878-3abc-4feb-9e9e-dc3104f8b4e5 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:59.501155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02edd17a-c34c-4c6b-aebf-3811afab06a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:59.501966 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edd17a-c34c-4c6b-aebf-3811afab06a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:59.502348 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edd17a-c34c-4c6b-aebf-3811afab06a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:59.502721 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edd17a-c34c-4c6b-aebf-3811afab06a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:59.504792 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-02edd17a-c34c-4c6b-aebf-3811afab06a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:59.523880 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328460be-3d42-412e-a745-4cc4e169187f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:59.530761 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfe496-cdd0-4f10-b0c2-0fe064940755 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eceda8b3a8ba4233851d0138d8f4d06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:59.531748 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfe496-cdd0-4f10-b0c2-0fe064940755 service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:59.532040 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfe496-cdd0-4f10-b0c2-0fe064940755 service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:59.532326 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfe496-cdd0-4f10-b0c2-0fe064940755 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:59.542304 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfe496-cdd0-4f10-b0c2-0fe064940755 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:59.544059 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfe496-cdd0-4f10-b0c2-0fe064940755 service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:59.552684 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1175/2347] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Oct 17 03:18:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:18:59.553506 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfe496-cdd0-4f10-b0c2-0fe064940755 service swift] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:18:59.554954 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-abcfe496-cdd0-4f10-b0c2-0fe064940755 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:18:59.595009 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:18:59.597661 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02edd17a-c34c-4c6b-aebf-3811afab06a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:59.599362 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02edd17a-c34c-4c6b-aebf-3811afab06a6 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:59.636672 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1173/2348] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:18:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:18:59.678494 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfe496-cdd0-4f10-b0c2-0fe064940755 service swift] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:18:59.694427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae77388-ad27-4538-8d92-f11e06c69e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:18:59.695779 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae77388-ad27-4538-8d92-f11e06c69e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:59.697839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae77388-ad27-4538-8d92-f11e06c69e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:59.699457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae77388-ad27-4538-8d92-f11e06c69e21 admin admin] PATH_INFO: `/v3/users/6e825afc2d614c338b37dbb7f8a04ec3` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:18:59.702662 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae77388-ad27-4538-8d92-f11e06c69e21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e825afc2d614c338b37dbb7f8a04ec3)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:18:59.708547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae77388-ad27-4538-8d92-f11e06c69e21 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:18:59.827513 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3ae77388-ad27-4538-8d92-f11e06c69e21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e825afc2d614c338b37dbb7f8a04ec3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:59.844207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3ae77388-ad27-4538-8d92-f11e06c69e21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e825afc2d614c338b37dbb7f8a04ec3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:59.846603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3ae77388-ad27-4538-8d92-f11e06c69e21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e825afc2d614c338b37dbb7f8a04ec3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:59.886142 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3ae77388-ad27-4538-8d92-f11e06c69e21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e825afc2d614c338b37dbb7f8a04ec3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:59.943054 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3ae77388-ad27-4538-8d92-f11e06c69e21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e825afc2d614c338b37dbb7f8a04ec3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:18:59.967491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1176/2349] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:18:59 2022] DELETE /identity/v3/users/6e825afc2d614c338b37dbb7f8a04ec3 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:18:59.968827 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd85ae-1bfa-492d-9372-238026ddb70a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:18:59.969216 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd85ae-1bfa-492d-9372-238026ddb70a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:18:59.970424 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd85ae-1bfa-492d-9372-238026ddb70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:00.022795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-92dd85ae-1bfa-492d-9372-238026ddb70a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:00.024170 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-92dd85ae-1bfa-492d-9372-238026ddb70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:00.051582 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae77388-ad27-4538-8d92-f11e06c69e21 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:00.053284 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3ae77388-ad27-4538-8d92-f11e06c69e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:00.144863 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:00.210336 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1174/2350] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:00.223396 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd85ae-1bfa-492d-9372-238026ddb70a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:00.225165 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-92dd85ae-1bfa-492d-9372-238026ddb70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:00.235470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae77388-ad27-4538-8d92-f11e06c69e21 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:00.244563 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b809dc-eeba-42ba-ad4b-4283b4eae002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:00.248193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b809dc-eeba-42ba-ad4b-4283b4eae002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:00.249224 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b809dc-eeba-42ba-ad4b-4283b4eae002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:00.250203 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b809dc-eeba-42ba-ad4b-4283b4eae002 admin admin] PATH_INFO: `/v3/projects/a8cad99a1f044f75824cc3c356308002` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:00.252403 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b809dc-eeba-42ba-ad4b-4283b4eae002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8cad99a1f044f75824cc3c356308002)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:00.255732 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b809dc-eeba-42ba-ad4b-4283b4eae002 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:00.356644 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd85ae-1bfa-492d-9372-238026ddb70a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:00.362905 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-22b809dc-eeba-42ba-ad4b-4283b4eae002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8cad99a1f044f75824cc3c356308002', 'request_id': 'req-22b809dc-eeba-42ba-ad4b-4283b4eae002'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:00.370723 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-22b809dc-eeba-42ba-ad4b-4283b4eae002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8cad99a1f044f75824cc3c356308002', 'request_id': 'req-22b809dc-eeba-42ba-ad4b-4283b4eae002'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:00.394234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-22b809dc-eeba-42ba-ad4b-4283b4eae002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8cad99a1f044f75824cc3c356308002', 'request_id': 'req-22b809dc-eeba-42ba-ad4b-4283b4eae002'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:00.429243 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-22b809dc-eeba-42ba-ad4b-4283b4eae002 admin admin] The token cache is being invalidate because project a8cad99a1f044f75824cc3c356308002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:19:00.429718 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-22b809dc-eeba-42ba-ad4b-4283b4eae002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:00.431691 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-22b809dc-eeba-42ba-ad4b-4283b4eae002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:00.441099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1177/2351] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:19:00 2022] DELETE /identity/v3/projects/a8cad99a1f044f75824cc3c356308002 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:19:00.446574 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a94e5-2c90-4492-bc96-8acfe4e1c3ed None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:00.446900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a94e5-2c90-4492-bc96-8acfe4e1c3ed None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:00.447171 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a94e5-2c90-4492-bc96-8acfe4e1c3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:00.478385 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20615c4f-8091-424d-8b09-2b828bc0ca96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:00.481809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20615c4f-8091-424d-8b09-2b828bc0ca96 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:00.482130 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20615c4f-8091-424d-8b09-2b828bc0ca96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:00.482381 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20615c4f-8091-424d-8b09-2b828bc0ca96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:00.493950 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-20615c4f-8091-424d-8b09-2b828bc0ca96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:00.510581 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-562a94e5-2c90-4492-bc96-8acfe4e1c3ed None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:00.513447 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-562a94e5-2c90-4492-bc96-8acfe4e1c3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:00.626132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20615c4f-8091-424d-8b09-2b828bc0ca96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:00.628692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20615c4f-8091-424d-8b09-2b828bc0ca96 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:00.637537 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1178/2352] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:00.644167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42ea90-21a1-4c26-8369-cddcc41bf56a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:00.644491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42ea90-21a1-4c26-8369-cddcc41bf56a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:00.645773 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42ea90-21a1-4c26-8369-cddcc41bf56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:00.673101 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1175/2353] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:19:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:19:00.706775 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-5f42ea90-21a1-4c26-8369-cddcc41bf56a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:00.711432 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-5f42ea90-21a1-4c26-8369-cddcc41bf56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:00.731822 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1179/2354] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:19:00.736098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20615c4f-8091-424d-8b09-2b828bc0ca96 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:00.741349 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=100 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:00.749155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20615c4f-8091-424d-8b09-2b828bc0ca96 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:00.764464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805b8a8-9680-42de-8337-de08cf65d92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:00.767914 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805b8a8-9680-42de-8337-de08cf65d92d admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:00.768376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805b8a8-9680-42de-8337-de08cf65d92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:00.768668 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805b8a8-9680-42de-8337-de08cf65d92d admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:00.769401 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805b8a8-9680-42de-8337-de08cf65d92d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:19:00.771348 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805b8a8-9680-42de-8337-de08cf65d92d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:00.773345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805b8a8-9680-42de-8337-de08cf65d92d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:00.805112 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1176/2355] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:19:00.809741 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42ea90-21a1-4c26-8369-cddcc41bf56a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:00.824774 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42ea90-21a1-4c26-8369-cddcc41bf56a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:00.836540 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421b913-10dd-4819-9018-9c2613bc2bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:00.837315 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b913-10dd-4819-9018-9c2613bc2bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:00.837728 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b913-10dd-4819-9018-9c2613bc2bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:00.838725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b913-10dd-4819-9018-9c2613bc2bd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:00.839610 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421b913-10dd-4819-9018-9c2613bc2bd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:00.841761 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421b913-10dd-4819-9018-9c2613bc2bd3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:00.901457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1180/2356] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:19:00 2022] POST /identity/v3/projects => generated 369 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:19:00.905585 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805b8a8-9680-42de-8337-de08cf65d92d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:00.923168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805b8a8-9680-42de-8337-de08cf65d92d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:01.107683 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be862eb-f80c-48d8-8c23-dcbd72c70e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:01.108208 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be862eb-f80c-48d8-8c23-dcbd72c70e59 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:01.108591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be862eb-f80c-48d8-8c23-dcbd72c70e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:01.108984 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be862eb-f80c-48d8-8c23-dcbd72c70e59 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:01.111030 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be862eb-f80c-48d8-8c23-dcbd72c70e59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:01.113948 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be862eb-f80c-48d8-8c23-dcbd72c70e59 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:01.169587 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1177/2357] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:19:00 2022] POST /identity/v3/users => generated 285 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:19:01.170554 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421b913-10dd-4819-9018-9c2613bc2bd3 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:01.184707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be862eb-f80c-48d8-8c23-dcbd72c70e59 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:01.186736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0be862eb-f80c-48d8-8c23-dcbd72c70e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:01.197149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421b913-10dd-4819-9018-9c2613bc2bd3 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:01.206399 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be862eb-f80c-48d8-8c23-dcbd72c70e59 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:01.211781 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:01.212601 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:01.213934 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:01.213934 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae admin admin] PATH_INFO: `/v3/projects/a3740bdc97e241bba60d94020ff51748` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:01.215305 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3740bdc97e241bba60d94020ff51748)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:01.217816 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:01.221675 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed55fc-f72c-40f1-aece-7e56f5c1915a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:01.222292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed55fc-f72c-40f1-aece-7e56f5c1915a admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:01.222569 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed55fc-f72c-40f1-aece-7e56f5c1915a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:01.222838 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed55fc-f72c-40f1-aece-7e56f5c1915a admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:01.226078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed55fc-f72c-40f1-aece-7e56f5c1915a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:01.230443 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed55fc-f72c-40f1-aece-7e56f5c1915a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:01.263143 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1181/2358] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:19:01.264911 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed55fc-f72c-40f1-aece-7e56f5c1915a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:01.287613 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed55fc-f72c-40f1-aece-7e56f5c1915a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:01.305318 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c39102-9bc4-4f8a-b3fc-25f886d73998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:01.311139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c39102-9bc4-4f8a-b3fc-25f886d73998 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:01.311575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c39102-9bc4-4f8a-b3fc-25f886d73998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:01.312317 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c39102-9bc4-4f8a-b3fc-25f886d73998 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:01.313603 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c39102-9bc4-4f8a-b3fc-25f886d73998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:01.318068 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c39102-9bc4-4f8a-b3fc-25f886d73998 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:01.355998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1182/2359] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:01.361758 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3740bdc97e241bba60d94020ff51748', 'request_id': 'req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:01.377326 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c39102-9bc4-4f8a-b3fc-25f886d73998 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:01.390315 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3740bdc97e241bba60d94020ff51748', 'request_id': 'req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:01.401131 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c39102-9bc4-4f8a-b3fc-25f886d73998 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:01.405363 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3740bdc97e241bba60d94020ff51748', 'request_id': 'req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:01.406782 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:01.440122 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae admin admin] The token cache is being invalidate because project a3740bdc97e241bba60d94020ff51748 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:19:01.440718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:01.443286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:01.446202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1178/2360] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:19:01 2022] DELETE /identity/v3/projects/a3740bdc97e241bba60d94020ff51748 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:19:01.450403 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:01.452020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:01.583211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b17ebc-840e-4d9e-8b6d-f58449dbe4ae admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:01.598699 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f97fd-403f-4ff7-858f-57e262a6b4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:01.599681 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f97fd-403f-4ff7-858f-57e262a6b4d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:01.600124 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f97fd-403f-4ff7-858f-57e262a6b4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:01.600549 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f97fd-403f-4ff7-858f-57e262a6b4d9 admin admin] PATH_INFO: `/v3/users/64a3290ba445439389d03f8803112c9f` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:01.601997 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f97fd-403f-4ff7-858f-57e262a6b4d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64a3290ba445439389d03f8803112c9f)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:01.604664 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f97fd-403f-4ff7-858f-57e262a6b4d9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:01.611404 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:01.636026 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b9764-8a5a-4062-ae7c-01b284929359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:01.637450 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b9764-8a5a-4062-ae7c-01b284929359 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:01.637818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b9764-8a5a-4062-ae7c-01b284929359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:01.638515 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b9764-8a5a-4062-ae7c-01b284929359 admin admin] PATH_INFO: `/v3/projects/73ec65cc06544a6d86bb4059de5d3d67/users/5c4bcd13733e499ab0a6267675d5acd5/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:01.673497 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b9764-8a5a-4062-ae7c-01b284929359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ec65cc06544a6d86bb4059de5d3d67, user_id=5c4bcd13733e499ab0a6267675d5acd5, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:01.679644 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b9764-8a5a-4062-ae7c-01b284929359 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:01.705684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fb0f97fd-403f-4ff7-858f-57e262a6b4d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64a3290ba445439389d03f8803112c9f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:01.720280 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1183/2361] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:19:01 2022] PUT /identity/v3/projects/73ec65cc06544a6d86bb4059de5d3d67/users/5c4bcd13733e499ab0a6267675d5acd5/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:01.721871 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f0aa0-d802-4c2c-9b89-45a7377bcaa4 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:01.722619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f0aa0-d802-4c2c-9b89-45a7377bcaa4 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:01.722619 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f0aa0-d802-4c2c-9b89-45a7377bcaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:01.735619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fb0f97fd-403f-4ff7-858f-57e262a6b4d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64a3290ba445439389d03f8803112c9f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:01.736985 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fb0f97fd-403f-4ff7-858f-57e262a6b4d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64a3290ba445439389d03f8803112c9f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:01.766001 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-533f0aa0-d802-4c2c-9b89-45a7377bcaa4 None None] MFA Rules not processed for user `5c4bcd13733e499ab0a6267675d5acd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:01.766994 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fb0f97fd-403f-4ff7-858f-57e262a6b4d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64a3290ba445439389d03f8803112c9f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:01.767774 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-533f0aa0-d802-4c2c-9b89-45a7377bcaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:01.805288 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fb0f97fd-403f-4ff7-858f-57e262a6b4d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64a3290ba445439389d03f8803112c9f'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:01.837747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1179/2362] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:19:01 2022] DELETE /identity/v3/users/64a3290ba445439389d03f8803112c9f => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:01.930903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1184/2363] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:01.955057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f97fd-403f-4ff7-858f-57e262a6b4d9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:01.956624 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fb0f97fd-403f-4ff7-858f-57e262a6b4d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:02.131373 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f97fd-403f-4ff7-858f-57e262a6b4d9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:02.147038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe8eb6-75c5-40a9-a772-a4b49f4d9aba service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8aa29639a3143d2a270e3248d2205d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:02.149847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe8eb6-75c5-40a9-a772-a4b49f4d9aba service glance] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:02.152084 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe8eb6-75c5-40a9-a772-a4b49f4d9aba service glance] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:02.152487 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe8eb6-75c5-40a9-a772-a4b49f4d9aba service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:02.301663 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe8eb6-75c5-40a9-a772-a4b49f4d9aba service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:02.305282 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe8eb6-75c5-40a9-a772-a4b49f4d9aba service glance] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:02.327215 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1180/2364] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:19:01 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:19:02.533383 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f0aa0-d802-4c2c-9b89-45a7377bcaa4 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:02.536569 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-533f0aa0-d802-4c2c-9b89-45a7377bcaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:02.629966 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f0aa0-d802-4c2c-9b89-45a7377bcaa4 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:02.668084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:02.728421 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466be2e-2db9-4a01-92d7-3e4c7e6b2fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:02.730203 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466be2e-2db9-4a01-92d7-3e4c7e6b2fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:02.730852 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466be2e-2db9-4a01-92d7-3e4c7e6b2fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:02.731371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466be2e-2db9-4a01-92d7-3e4c7e6b2fd0 admin admin] PATH_INFO: `/v3/projects/72b6366bd24846a297f5e4174cfe4eb4` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:02.732976 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466be2e-2db9-4a01-92d7-3e4c7e6b2fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72b6366bd24846a297f5e4174cfe4eb4)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:02.735422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466be2e-2db9-4a01-92d7-3e4c7e6b2fd0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:02.827020 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2466be2e-2db9-4a01-92d7-3e4c7e6b2fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72b6366bd24846a297f5e4174cfe4eb4', 'request_id': 'req-2466be2e-2db9-4a01-92d7-3e4c7e6b2fd0'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:02.835291 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2466be2e-2db9-4a01-92d7-3e4c7e6b2fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72b6366bd24846a297f5e4174cfe4eb4', 'request_id': 'req-2466be2e-2db9-4a01-92d7-3e4c7e6b2fd0'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:02.844951 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2466be2e-2db9-4a01-92d7-3e4c7e6b2fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72b6366bd24846a297f5e4174cfe4eb4', 'request_id': 'req-2466be2e-2db9-4a01-92d7-3e4c7e6b2fd0'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:02.871122 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2466be2e-2db9-4a01-92d7-3e4c7e6b2fd0 admin admin] The token cache is being invalidate because project 72b6366bd24846a297f5e4174cfe4eb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:19:02.871655 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2466be2e-2db9-4a01-92d7-3e4c7e6b2fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:02.872750 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2466be2e-2db9-4a01-92d7-3e4c7e6b2fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:02.876109 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1185/2365] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:19:02 2022] DELETE /identity/v3/projects/72b6366bd24846a297f5e4174cfe4eb4 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:02.881805 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2215e1-0f1e-43a5-9dfa-94bebd964f03 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:02.882531 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2215e1-0f1e-43a5-9dfa-94bebd964f03 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:02.882670 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2215e1-0f1e-43a5-9dfa-94bebd964f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:02.929918 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4d2215e1-0f1e-43a5-9dfa-94bebd964f03 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:02.932357 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4d2215e1-0f1e-43a5-9dfa-94bebd964f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:03.013112 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:03.087092 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1181/2366] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:03.096340 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce4604-0f40-4f12-ae45-3c328278a958 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:03.096799 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce4604-0f40-4f12-ae45-3c328278a958 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:03.097269 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce4604-0f40-4f12-ae45-3c328278a958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:03.151476 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-c9ce4604-0f40-4f12-ae45-3c328278a958 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:03.156269 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c9ce4604-0f40-4f12-ae45-3c328278a958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:03.183276 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1186/2367] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:03.187775 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2215e1-0f1e-43a5-9dfa-94bebd964f03 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:03.204201 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2215e1-0f1e-43a5-9dfa-94bebd964f03 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:03.224598 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e208f6e-8461-47eb-b6db-225d1fa7f8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:03.226731 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e208f6e-8461-47eb-b6db-225d1fa7f8c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:03.227975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e208f6e-8461-47eb-b6db-225d1fa7f8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:03.230124 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e208f6e-8461-47eb-b6db-225d1fa7f8c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:03.231088 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e208f6e-8461-47eb-b6db-225d1fa7f8c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:19:03.232888 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e208f6e-8461-47eb-b6db-225d1fa7f8c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:03.234729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e208f6e-8461-47eb-b6db-225d1fa7f8c9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:03.265197 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1182/2368] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:19:03.268678 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce4604-0f40-4f12-ae45-3c328278a958 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:03.296258 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce4604-0f40-4f12-ae45-3c328278a958 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:03.314784 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93491ee5-82e5-41ae-aebf-4669bd45db34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:03.315443 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93491ee5-82e5-41ae-aebf-4669bd45db34 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:03.315720 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93491ee5-82e5-41ae-aebf-4669bd45db34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:03.315993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93491ee5-82e5-41ae-aebf-4669bd45db34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:03.316639 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93491ee5-82e5-41ae-aebf-4669bd45db34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:03.318555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93491ee5-82e5-41ae-aebf-4669bd45db34 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:03.381197 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1187/2369] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:19:03 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:19:03.385617 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e208f6e-8461-47eb-b6db-225d1fa7f8c9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:03.399711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e208f6e-8461-47eb-b6db-225d1fa7f8c9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:03.574504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1a6aa-9bc8-43d0-9139-1e8117680b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:03.575296 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1a6aa-9bc8-43d0-9139-1e8117680b8f admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:03.575580 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1a6aa-9bc8-43d0-9139-1e8117680b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:03.575860 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1a6aa-9bc8-43d0-9139-1e8117680b8f admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:03.576507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1a6aa-9bc8-43d0-9139-1e8117680b8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:03.578637 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1a6aa-9bc8-43d0-9139-1e8117680b8f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:03.610688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1183/2370] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:19:03 2022] POST /identity/v3/users => generated 284 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:19:03.612333 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93491ee5-82e5-41ae-aebf-4669bd45db34 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:03.625593 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93491ee5-82e5-41ae-aebf-4669bd45db34 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:03.633867 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e8d82-bcfc-4d5e-b83b-2ad643121132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:03.634982 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e8d82-bcfc-4d5e-b83b-2ad643121132 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:03.636077 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e8d82-bcfc-4d5e-b83b-2ad643121132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:03.636583 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e8d82-bcfc-4d5e-b83b-2ad643121132 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:03.637560 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e8d82-bcfc-4d5e-b83b-2ad643121132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:03.640700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e8d82-bcfc-4d5e-b83b-2ad643121132 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:03.660479 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1a6aa-9bc8-43d0-9139-1e8117680b8f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:03.668730 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1188/2371] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:03.676892 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1a6aa-9bc8-43d0-9139-1e8117680b8f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:03.683438 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c3c98-a863-4131-b738-0d3503c0a2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:03.684213 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c3c98-a863-4131-b738-0d3503c0a2a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:03.684641 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c3c98-a863-4131-b738-0d3503c0a2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:03.684992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c3c98-a863-4131-b738-0d3503c0a2a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:03.685776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c3c98-a863-4131-b738-0d3503c0a2a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:03.687872 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c3c98-a863-4131-b738-0d3503c0a2a4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:03.706232 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e8d82-bcfc-4d5e-b83b-2ad643121132 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:03.712037 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1184/2372] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:19:03.718859 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e8d82-bcfc-4d5e-b83b-2ad643121132 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:03.728854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46d7b6-6acf-4263-bab1-6369a83a20db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:03.729646 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46d7b6-6acf-4263-bab1-6369a83a20db admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:03.730036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46d7b6-6acf-4263-bab1-6369a83a20db admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:03.731371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46d7b6-6acf-4263-bab1-6369a83a20db admin admin] PATH_INFO: `/v3/projects/d3fa9697518749bfbe0b4297b824f4e5/users/d503e3ffa1be49d692deb0b38a37e5da/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:03.755297 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46d7b6-6acf-4263-bab1-6369a83a20db admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3fa9697518749bfbe0b4297b824f4e5, user_id=d503e3ffa1be49d692deb0b38a37e5da, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:03.758253 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46d7b6-6acf-4263-bab1-6369a83a20db admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:03.786270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1189/2373] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:19:03 2022] PUT /identity/v3/projects/d3fa9697518749bfbe0b4297b824f4e5/users/d503e3ffa1be49d692deb0b38a37e5da/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:03.788326 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5f5da-eb38-47d4-b670-314b4631688c None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:03.788899 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5f5da-eb38-47d4-b670-314b4631688c None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:03.789249 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb5f5da-eb38-47d4-b670-314b4631688c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:03.834635 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-dfb5f5da-eb38-47d4-b670-314b4631688c None None] MFA Rules not processed for user `d503e3ffa1be49d692deb0b38a37e5da`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:03.836087 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dfb5f5da-eb38-47d4-b670-314b4631688c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:03.953069 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1185/2374] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:19:08.109584 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46d7b6-6acf-4263-bab1-6369a83a20db admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:08.111499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-1a46d7b6-6acf-4263-bab1-6369a83a20db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:08.184984 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46d7b6-6acf-4263-bab1-6369a83a20db admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:08.189137 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614e779-3391-4325-8dcd-b9ddd75ff538 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8aa29639a3143d2a270e3248d2205d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:08.189910 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614e779-3391-4325-8dcd-b9ddd75ff538 service glance] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:08.190348 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614e779-3391-4325-8dcd-b9ddd75ff538 service glance] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:08.190761 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614e779-3391-4325-8dcd-b9ddd75ff538 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:08.201622 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:08.205821 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614e779-3391-4325-8dcd-b9ddd75ff538 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:08.207564 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614e779-3391-4325-8dcd-b9ddd75ff538 service glance] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:08.227093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1190/2375] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:19:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:19:10.920255 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5f5da-eb38-47d4-b670-314b4631688c None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:10.921831 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-dfb5f5da-eb38-47d4-b670-314b4631688c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:10.956103 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:10.990135 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb5f5da-eb38-47d4-b670-314b4631688c None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:10.994595 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133f06a-be9b-43fd-b14f-49215f82849b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:10.995256 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133f06a-be9b-43fd-b14f-49215f82849b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:10.995534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133f06a-be9b-43fd-b14f-49215f82849b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:10.995806 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133f06a-be9b-43fd-b14f-49215f82849b admin admin] PATH_INFO: `/v3/users/3862054580664f8494c5583ee8334ecb` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:10.996949 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2133f06a-be9b-43fd-b14f-49215f82849b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3862054580664f8494c5583ee8334ecb)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:10.998851 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2133f06a-be9b-43fd-b14f-49215f82849b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:11.045510 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2133f06a-be9b-43fd-b14f-49215f82849b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3862054580664f8494c5583ee8334ecb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:11.052294 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2133f06a-be9b-43fd-b14f-49215f82849b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3862054580664f8494c5583ee8334ecb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:11.052947 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2133f06a-be9b-43fd-b14f-49215f82849b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3862054580664f8494c5583ee8334ecb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:11.060507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2133f06a-be9b-43fd-b14f-49215f82849b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3862054580664f8494c5583ee8334ecb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:11.072371 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-2133f06a-be9b-43fd-b14f-49215f82849b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3862054580664f8494c5583ee8334ecb'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:11.078682 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1186/2376] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:19:10 2022] DELETE /identity/v3/users/3862054580664f8494c5583ee8334ecb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:11.083191 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d32368-9d0b-43cb-ac75-1844c2d4f229 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:11.083645 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d32368-9d0b-43cb-ac75-1844c2d4f229 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:11.083755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d32368-9d0b-43cb-ac75-1844c2d4f229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:11.102807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-b8d32368-9d0b-43cb-ac75-1844c2d4f229 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:11.104599 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b8d32368-9d0b-43cb-ac75-1844c2d4f229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:11.162857 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1191/2377] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:11.173683 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133f06a-be9b-43fd-b14f-49215f82849b admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:11.175160 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2133f06a-be9b-43fd-b14f-49215f82849b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:11.237780 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133f06a-be9b-43fd-b14f-49215f82849b admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:11.241475 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be542ed2-f5cb-4bb2-a7d2-26c56d738f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:11.242087 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be542ed2-f5cb-4bb2-a7d2-26c56d738f30 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:11.242367 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be542ed2-f5cb-4bb2-a7d2-26c56d738f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:11.242641 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be542ed2-f5cb-4bb2-a7d2-26c56d738f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:11.252600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be542ed2-f5cb-4bb2-a7d2-26c56d738f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:11.254028 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be542ed2-f5cb-4bb2-a7d2-26c56d738f30 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:11.263911 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1187/2378] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:19:11.530760 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d32368-9d0b-43cb-ac75-1844c2d4f229 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:11.532244 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b8d32368-9d0b-43cb-ac75-1844c2d4f229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:11.545207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d32368-9d0b-43cb-ac75-1844c2d4f229 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:11.551235 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:11.552327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:11.552835 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:11.553189 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853 admin admin] PATH_INFO: `/v3/projects/6d623783bda1452aa94058ca8ea1e2b3` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:11.554958 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d623783bda1452aa94058ca8ea1e2b3)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:11.558354 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:11.626926 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d623783bda1452aa94058ca8ea1e2b3', 'request_id': 'req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:11.632657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d623783bda1452aa94058ca8ea1e2b3', 'request_id': 'req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:11.641036 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d623783bda1452aa94058ca8ea1e2b3', 'request_id': 'req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:11.656093 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853 admin admin] The token cache is being invalidate because project 6d623783bda1452aa94058ca8ea1e2b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:19:11.656387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:11.657072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:11.661553 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1192/2379] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:19:11 2022] DELETE /identity/v3/projects/6d623783bda1452aa94058ca8ea1e2b3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:11.663213 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be542ed2-f5cb-4bb2-a7d2-26c56d738f30 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:11.674923 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be542ed2-f5cb-4bb2-a7d2-26c56d738f30 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:11.736502 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:11.751535 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be190744-5e06-4202-a7d4-f8011edd166a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:11.752710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be190744-5e06-4202-a7d4-f8011edd166a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:11.753067 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be190744-5e06-4202-a7d4-f8011edd166a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:11.754224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be190744-5e06-4202-a7d4-f8011edd166a admin admin] PATH_INFO: `/v3/users/c0074a560a0642e8bf7c0435cec5bb15` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:11.756718 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be190744-5e06-4202-a7d4-f8011edd166a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0074a560a0642e8bf7c0435cec5bb15)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:11.759271 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be190744-5e06-4202-a7d4-f8011edd166a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:11.838913 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-be190744-5e06-4202-a7d4-f8011edd166a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0074a560a0642e8bf7c0435cec5bb15'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:11.846258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-be190744-5e06-4202-a7d4-f8011edd166a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0074a560a0642e8bf7c0435cec5bb15'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:11.847028 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-be190744-5e06-4202-a7d4-f8011edd166a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0074a560a0642e8bf7c0435cec5bb15'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:11.855869 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-be190744-5e06-4202-a7d4-f8011edd166a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0074a560a0642e8bf7c0435cec5bb15'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:11.871067 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-be190744-5e06-4202-a7d4-f8011edd166a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0074a560a0642e8bf7c0435cec5bb15'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:11.888985 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1188/2380] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:19:11 2022] DELETE /identity/v3/users/c0074a560a0642e8bf7c0435cec5bb15 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:12.246920 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:12.248384 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:12.262745 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d0f9c-0d57-42c8-9f64-2a3a21736853 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:12.331233 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdcf6b-941a-4387-b034-673bbc441318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:12.331898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdcf6b-941a-4387-b034-673bbc441318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:12.332185 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdcf6b-941a-4387-b034-673bbc441318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:12.332465 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdcf6b-941a-4387-b034-673bbc441318 admin admin] PATH_INFO: `/v3/projects/b2a1f59ad2ea49b2bfe6a2887ccd3494` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:12.333829 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdcf6b-941a-4387-b034-673bbc441318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2a1f59ad2ea49b2bfe6a2887ccd3494)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:12.335832 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdcf6b-941a-4387-b034-673bbc441318 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:12.363837 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:12.390101 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-68cdcf6b-941a-4387-b034-673bbc441318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2a1f59ad2ea49b2bfe6a2887ccd3494', 'request_id': 'req-68cdcf6b-941a-4387-b034-673bbc441318'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:12.396101 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-68cdcf6b-941a-4387-b034-673bbc441318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2a1f59ad2ea49b2bfe6a2887ccd3494', 'request_id': 'req-68cdcf6b-941a-4387-b034-673bbc441318'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:12.402954 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-68cdcf6b-941a-4387-b034-673bbc441318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2a1f59ad2ea49b2bfe6a2887ccd3494', 'request_id': 'req-68cdcf6b-941a-4387-b034-673bbc441318'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:12.415141 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-68cdcf6b-941a-4387-b034-673bbc441318 admin admin] The token cache is being invalidate because project b2a1f59ad2ea49b2bfe6a2887ccd3494 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:19:12.415428 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-68cdcf6b-941a-4387-b034-673bbc441318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:12.416119 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-68cdcf6b-941a-4387-b034-673bbc441318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:12.417874 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1193/2381] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:19:12 2022] DELETE /identity/v3/projects/b2a1f59ad2ea49b2bfe6a2887ccd3494 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:12.423645 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766de4e-2883-49ca-8e8e-2433559f7caf None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:12.424117 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766de4e-2883-49ca-8e8e-2433559f7caf None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:12.424214 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766de4e-2883-49ca-8e8e-2433559f7caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:12.445612 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-0766de4e-2883-49ca-8e8e-2433559f7caf None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:12.446833 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-0766de4e-2883-49ca-8e8e-2433559f7caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:12.507297 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1189/2382] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:12.516062 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4a408-ffec-48c9-ac80-bc7e93a2f87c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:12.516292 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4a408-ffec-48c9-ac80-bc7e93a2f87c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:12.516569 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4a408-ffec-48c9-ac80-bc7e93a2f87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:12.537232 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-dec4a408-ffec-48c9-ac80-bc7e93a2f87c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:12.538483 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-dec4a408-ffec-48c9-ac80-bc7e93a2f87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:12.546673 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1194/2383] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:12.551106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766de4e-2883-49ca-8e8e-2433559f7caf None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:12.561306 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766de4e-2883-49ca-8e8e-2433559f7caf None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:12.565852 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cba8a9-7e57-4325-b032-2ef886d90b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:12.566446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cba8a9-7e57-4325-b032-2ef886d90b09 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:12.566722 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cba8a9-7e57-4325-b032-2ef886d90b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:12.567034 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cba8a9-7e57-4325-b032-2ef886d90b09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:12.567714 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cba8a9-7e57-4325-b032-2ef886d90b09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:19:12.567997 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cba8a9-7e57-4325-b032-2ef886d90b09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:12.569564 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cba8a9-7e57-4325-b032-2ef886d90b09 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:12.581343 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1190/2384] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:19:12.585812 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4a408-ffec-48c9-ac80-bc7e93a2f87c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:12.596306 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4a408-ffec-48c9-ac80-bc7e93a2f87c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:12.601135 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7cd73a-e836-4e31-aa3b-b1a7833ebf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:12.601746 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cd73a-e836-4e31-aa3b-b1a7833ebf59 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:12.602024 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cd73a-e836-4e31-aa3b-b1a7833ebf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:12.602297 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cd73a-e836-4e31-aa3b-b1a7833ebf59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:12.602960 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7cd73a-e836-4e31-aa3b-b1a7833ebf59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:12.604889 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7cd73a-e836-4e31-aa3b-b1a7833ebf59 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:12.631600 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1195/2385] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:19:12 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:19:12.636700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cba8a9-7e57-4325-b032-2ef886d90b09 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:12.646397 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cba8a9-7e57-4325-b032-2ef886d90b09 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:12.704650 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5d2b8-147b-4e62-a89c-4b6416e788d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:12.705273 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5d2b8-147b-4e62-a89c-4b6416e788d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:12.705552 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5d2b8-147b-4e62-a89c-4b6416e788d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:12.705823 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5d2b8-147b-4e62-a89c-4b6416e788d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:12.706533 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5d2b8-147b-4e62-a89c-4b6416e788d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:12.708647 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5d2b8-147b-4e62-a89c-4b6416e788d9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:12.716976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:12.734937 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1191/2386] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:19:12 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:19:12.736075 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7cd73a-e836-4e31-aa3b-b1a7833ebf59 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:12.747760 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7cd73a-e836-4e31-aa3b-b1a7833ebf59 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:12.752420 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c1716-48c8-44d0-a8b8-0035456fa54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:12.753121 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c1716-48c8-44d0-a8b8-0035456fa54c admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:12.753467 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c1716-48c8-44d0-a8b8-0035456fa54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:12.753805 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c1716-48c8-44d0-a8b8-0035456fa54c admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:12.754605 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c1716-48c8-44d0-a8b8-0035456fa54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:12.756321 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c1716-48c8-44d0-a8b8-0035456fa54c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:12.767499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1196/2387] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:12.771181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5d2b8-147b-4e62-a89c-4b6416e788d9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:12.781700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5d2b8-147b-4e62-a89c-4b6416e788d9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:12.786538 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea88c80-43da-4b68-ac13-e34503d1a281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:12.787429 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea88c80-43da-4b68-ac13-e34503d1a281 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:12.787717 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea88c80-43da-4b68-ac13-e34503d1a281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:12.787991 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea88c80-43da-4b68-ac13-e34503d1a281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:12.788636 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea88c80-43da-4b68-ac13-e34503d1a281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:12.790234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea88c80-43da-4b68-ac13-e34503d1a281 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:12.804098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1192/2388] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:12.806259 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c1716-48c8-44d0-a8b8-0035456fa54c admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:12.818247 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c1716-48c8-44d0-a8b8-0035456fa54c admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:12.824072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610c5fe-932b-48e1-bf52-1565e3b7f9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:12.824733 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610c5fe-932b-48e1-bf52-1565e3b7f9be admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:12.825018 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610c5fe-932b-48e1-bf52-1565e3b7f9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:12.825294 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610c5fe-932b-48e1-bf52-1565e3b7f9be admin admin] PATH_INFO: `/v3/projects/6e1ddb1187d44dbbb5632d17d22ac3d7/users/ba518a2c8a4447cf845626380f35abcc/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:12.843477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610c5fe-932b-48e1-bf52-1565e3b7f9be admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1ddb1187d44dbbb5632d17d22ac3d7, user_id=ba518a2c8a4447cf845626380f35abcc, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:12.848591 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610c5fe-932b-48e1-bf52-1565e3b7f9be admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:12.867767 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1197/2389] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:19:12 2022] PUT /identity/v3/projects/6e1ddb1187d44dbbb5632d17d22ac3d7/users/ba518a2c8a4447cf845626380f35abcc/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:12.878609 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96e9df-ffd4-4b76-bdf0-a462e57a003d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:12.878919 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96e9df-ffd4-4b76-bdf0-a462e57a003d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:12.879196 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96e9df-ffd4-4b76-bdf0-a462e57a003d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:12.907117 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-6f96e9df-ffd4-4b76-bdf0-a462e57a003d None None] MFA Rules not processed for user `ba518a2c8a4447cf845626380f35abcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:12.908323 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-6f96e9df-ffd4-4b76-bdf0-a462e57a003d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:12.973932 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1193/2390] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:12.978391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610c5fe-932b-48e1-bf52-1565e3b7f9be admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:13.045428 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610c5fe-932b-48e1-bf52-1565e3b7f9be admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:13.050550 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f34c1-5f32-43a6-8ad1-fe35123e1550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:13.051218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f34c1-5f32-43a6-8ad1-fe35123e1550 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:13.051503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f34c1-5f32-43a6-8ad1-fe35123e1550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:13.051776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f34c1-5f32-43a6-8ad1-fe35123e1550 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:13.052457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f34c1-5f32-43a6-8ad1-fe35123e1550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:13.054413 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f34c1-5f32-43a6-8ad1-fe35123e1550 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:13.082944 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1198/2391] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:19:12 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:19:13.087075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96e9df-ffd4-4b76-bdf0-a462e57a003d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:13.098344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f96e9df-ffd4-4b76-bdf0-a462e57a003d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:13.155192 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0dadf-3d83-4d7c-87cb-024ea237b138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:13.155800 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0dadf-3d83-4d7c-87cb-024ea237b138 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:13.156077 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0dadf-3d83-4d7c-87cb-024ea237b138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:13.156348 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0dadf-3d83-4d7c-87cb-024ea237b138 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:13.156980 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0dadf-3d83-4d7c-87cb-024ea237b138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:13.158897 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0dadf-3d83-4d7c-87cb-024ea237b138 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:13.180996 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1194/2392] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:19:13 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:19:13.185025 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f34c1-5f32-43a6-8ad1-fe35123e1550 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:13.195253 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f34c1-5f32-43a6-8ad1-fe35123e1550 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:13.200103 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7945ff4-7fd1-4fd2-8920-03a0314cce1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:13.200699 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7945ff4-7fd1-4fd2-8920-03a0314cce1d admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:13.200981 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7945ff4-7fd1-4fd2-8920-03a0314cce1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:13.201255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7945ff4-7fd1-4fd2-8920-03a0314cce1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:13.201893 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7945ff4-7fd1-4fd2-8920-03a0314cce1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:13.203527 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7945ff4-7fd1-4fd2-8920-03a0314cce1d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:13.214839 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1199/2393] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:13.218779 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0dadf-3d83-4d7c-87cb-024ea237b138 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:13.229183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0dadf-3d83-4d7c-87cb-024ea237b138 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:13.233723 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801b0f4-76aa-47cd-a822-e4c778f3a3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:13.234370 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801b0f4-76aa-47cd-a822-e4c778f3a3fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:13.234649 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801b0f4-76aa-47cd-a822-e4c778f3a3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:13.234936 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801b0f4-76aa-47cd-a822-e4c778f3a3fa admin admin] PATH_INFO: `/v3/projects/1a8e9dddc3b841508f5878a72d22081e/users/7b213ef921fd431585679eeca92b5a6e/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:13.249656 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801b0f4-76aa-47cd-a822-e4c778f3a3fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a8e9dddc3b841508f5878a72d22081e, user_id=7b213ef921fd431585679eeca92b5a6e, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:13.252380 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801b0f4-76aa-47cd-a822-e4c778f3a3fa admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:13.265033 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1195/2394] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:19:13 2022] PUT /identity/v3/projects/1a8e9dddc3b841508f5878a72d22081e/users/7b213ef921fd431585679eeca92b5a6e/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:13.269122 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7945ff4-7fd1-4fd2-8920-03a0314cce1d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:13.279842 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7945ff4-7fd1-4fd2-8920-03a0314cce1d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:13.298434 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:13.340120 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5dcc2a-d7fd-4808-86fc-b1595b9e89a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:13.340727 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dcc2a-d7fd-4808-86fc-b1595b9e89a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:13.341008 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dcc2a-d7fd-4808-86fc-b1595b9e89a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:13.341283 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dcc2a-d7fd-4808-86fc-b1595b9e89a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:13.341923 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5dcc2a-d7fd-4808-86fc-b1595b9e89a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:13.343842 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5dcc2a-d7fd-4808-86fc-b1595b9e89a0 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:13.355178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1200/2395] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:13.359398 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801b0f4-76aa-47cd-a822-e4c778f3a3fa admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:13.373757 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801b0f4-76aa-47cd-a822-e4c778f3a3fa admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:13.377300 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55641e-46b0-4df1-848c-34071e1912de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:13.378137 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55641e-46b0-4df1-848c-34071e1912de admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:13.378319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55641e-46b0-4df1-848c-34071e1912de admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:13.378600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55641e-46b0-4df1-848c-34071e1912de admin admin] PATH_INFO: `/v3/projects/1a8e9dddc3b841508f5878a72d22081e/users/7b213ef921fd431585679eeca92b5a6e/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:13.381222 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55641e-46b0-4df1-848c-34071e1912de admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a8e9dddc3b841508f5878a72d22081e, user_id=7b213ef921fd431585679eeca92b5a6e, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:13.384597 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55641e-46b0-4df1-848c-34071e1912de admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:13.397726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1196/2396] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:19:13 2022] PUT /identity/v3/projects/1a8e9dddc3b841508f5878a72d22081e/users/7b213ef921fd431585679eeca92b5a6e/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:13.410148 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae79aa-f8c8-4889-82c1-7e5e57e8b875 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:13.410429 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae79aa-f8c8-4889-82c1-7e5e57e8b875 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:13.410706 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae79aa-f8c8-4889-82c1-7e5e57e8b875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:13.442173 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-28ae79aa-f8c8-4889-82c1-7e5e57e8b875 None None] MFA Rules not processed for user `7b213ef921fd431585679eeca92b5a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:13.443418 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-28ae79aa-f8c8-4889-82c1-7e5e57e8b875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:13.512869 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1201/2397] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:13 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:13.524096 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55641e-46b0-4df1-848c-34071e1912de admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:13.526973 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-bf55641e-46b0-4df1-848c-34071e1912de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:13.550116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:13.588828 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55641e-46b0-4df1-848c-34071e1912de admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:13.592869 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874de2d-af93-4992-9081-340c8ec7e064 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:13.593488 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874de2d-af93-4992-9081-340c8ec7e064 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:13.593767 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874de2d-af93-4992-9081-340c8ec7e064 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:13.594039 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874de2d-af93-4992-9081-340c8ec7e064 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:13.604093 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874de2d-af93-4992-9081-340c8ec7e064 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:13.605759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874de2d-af93-4992-9081-340c8ec7e064 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:13.616425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1197/2398] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:19:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:19:15.164199 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae79aa-f8c8-4889-82c1-7e5e57e8b875 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:15.246070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae79aa-f8c8-4889-82c1-7e5e57e8b875 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:15.250558 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88a402-e664-4842-a009-fd9284f29658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:15.251204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88a402-e664-4842-a009-fd9284f29658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:15.251484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88a402-e664-4842-a009-fd9284f29658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:15.251769 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88a402-e664-4842-a009-fd9284f29658 admin admin] PATH_INFO: `/v3/users/ba518a2c8a4447cf845626380f35abcc` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:15.252852 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88a402-e664-4842-a009-fd9284f29658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba518a2c8a4447cf845626380f35abcc)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:15.254765 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88a402-e664-4842-a009-fd9284f29658 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:15.304802 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:15.308323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7c88a402-e664-4842-a009-fd9284f29658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba518a2c8a4447cf845626380f35abcc'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:15.316261 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7c88a402-e664-4842-a009-fd9284f29658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba518a2c8a4447cf845626380f35abcc'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:15.316931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7c88a402-e664-4842-a009-fd9284f29658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba518a2c8a4447cf845626380f35abcc'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:15.329781 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7c88a402-e664-4842-a009-fd9284f29658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba518a2c8a4447cf845626380f35abcc'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:15.342488 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-7c88a402-e664-4842-a009-fd9284f29658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba518a2c8a4447cf845626380f35abcc'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:15.348279 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1202/2399] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:19:15 2022] DELETE /identity/v3/users/ba518a2c8a4447cf845626380f35abcc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:15.352854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6f846-6f4c-44e7-989b-94fca5c449b8 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:15.353299 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6f846-6f4c-44e7-989b-94fca5c449b8 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:15.353411 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6f846-6f4c-44e7-989b-94fca5c449b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:15.374326 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-4ab6f846-6f4c-44e7-989b-94fca5c449b8 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:15.375595 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4ab6f846-6f4c-44e7-989b-94fca5c449b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:15.435590 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1198/2400] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:15.445241 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88a402-e664-4842-a009-fd9284f29658 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:15.446644 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7c88a402-e664-4842-a009-fd9284f29658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:15.505787 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88a402-e664-4842-a009-fd9284f29658 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:15.509519 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07352a63-fd46-4a51-8382-d22fa2397cb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:15.510726 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07352a63-fd46-4a51-8382-d22fa2397cb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:15.511032 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07352a63-fd46-4a51-8382-d22fa2397cb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:15.511295 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07352a63-fd46-4a51-8382-d22fa2397cb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:15.520914 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07352a63-fd46-4a51-8382-d22fa2397cb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:15.522379 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07352a63-fd46-4a51-8382-d22fa2397cb7 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:15.532291 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1203/2401] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:19:15.803250 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6f846-6f4c-44e7-989b-94fca5c449b8 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:15.804780 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4ab6f846-6f4c-44e7-989b-94fca5c449b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:15.817143 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6f846-6f4c-44e7-989b-94fca5c449b8 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:15.830936 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a322ce-b7af-4e43-87fc-6494d47b5c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:15.831815 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a322ce-b7af-4e43-87fc-6494d47b5c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:15.832312 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a322ce-b7af-4e43-87fc-6494d47b5c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:15.832736 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a322ce-b7af-4e43-87fc-6494d47b5c2a admin admin] PATH_INFO: `/v3/projects/6e1ddb1187d44dbbb5632d17d22ac3d7` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:15.837727 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a322ce-b7af-4e43-87fc-6494d47b5c2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e1ddb1187d44dbbb5632d17d22ac3d7)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:15.841090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a322ce-b7af-4e43-87fc-6494d47b5c2a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:15.888284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-02a322ce-b7af-4e43-87fc-6494d47b5c2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e1ddb1187d44dbbb5632d17d22ac3d7', 'request_id': 'req-02a322ce-b7af-4e43-87fc-6494d47b5c2a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:15.893646 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-02a322ce-b7af-4e43-87fc-6494d47b5c2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e1ddb1187d44dbbb5632d17d22ac3d7', 'request_id': 'req-02a322ce-b7af-4e43-87fc-6494d47b5c2a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:15.899896 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-02a322ce-b7af-4e43-87fc-6494d47b5c2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e1ddb1187d44dbbb5632d17d22ac3d7', 'request_id': 'req-02a322ce-b7af-4e43-87fc-6494d47b5c2a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:15.916770 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-02a322ce-b7af-4e43-87fc-6494d47b5c2a admin admin] The token cache is being invalidate because project 6e1ddb1187d44dbbb5632d17d22ac3d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:19:15.917053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-02a322ce-b7af-4e43-87fc-6494d47b5c2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:15.917753 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-02a322ce-b7af-4e43-87fc-6494d47b5c2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:15.919557 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1199/2402] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:19:15 2022] DELETE /identity/v3/projects/6e1ddb1187d44dbbb5632d17d22ac3d7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:15.923420 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07352a63-fd46-4a51-8382-d22fa2397cb7 service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:15.934345 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07352a63-fd46-4a51-8382-d22fa2397cb7 service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:15.992296 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad74da-6beb-4cb6-9a88-4f4d27dfacd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:15.992921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad74da-6beb-4cb6-9a88-4f4d27dfacd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:15.993202 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad74da-6beb-4cb6-9a88-4f4d27dfacd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:15.993469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad74da-6beb-4cb6-9a88-4f4d27dfacd3 admin admin] PATH_INFO: `/v3/users/7b213ef921fd431585679eeca92b5a6e` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:15.994672 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad74da-6beb-4cb6-9a88-4f4d27dfacd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b213ef921fd431585679eeca92b5a6e)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:15.996609 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad74da-6beb-4cb6-9a88-4f4d27dfacd3 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:16.048137 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-82ad74da-6beb-4cb6-9a88-4f4d27dfacd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b213ef921fd431585679eeca92b5a6e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:16.054930 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-82ad74da-6beb-4cb6-9a88-4f4d27dfacd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b213ef921fd431585679eeca92b5a6e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:16.055588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-82ad74da-6beb-4cb6-9a88-4f4d27dfacd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b213ef921fd431585679eeca92b5a6e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:16.063535 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-82ad74da-6beb-4cb6-9a88-4f4d27dfacd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b213ef921fd431585679eeca92b5a6e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:16.065425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:16.078818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-82ad74da-6beb-4cb6-9a88-4f4d27dfacd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b213ef921fd431585679eeca92b5a6e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:16.085401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1204/2403] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:19:15 2022] DELETE /identity/v3/users/7b213ef921fd431585679eeca92b5a6e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:16.386896 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a322ce-b7af-4e43-87fc-6494d47b5c2a admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:16.388317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-02a322ce-b7af-4e43-87fc-6494d47b5c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:16.403834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a322ce-b7af-4e43-87fc-6494d47b5c2a admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:16.457631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:16.472842 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad9199-1a2e-49d7-991a-149ef1542148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:16.473575 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad9199-1a2e-49d7-991a-149ef1542148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:16.473931 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad9199-1a2e-49d7-991a-149ef1542148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:16.474525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad9199-1a2e-49d7-991a-149ef1542148 admin admin] PATH_INFO: `/v3/projects/1a8e9dddc3b841508f5878a72d22081e` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:16.475605 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad9199-1a2e-49d7-991a-149ef1542148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a8e9dddc3b841508f5878a72d22081e)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:16.477635 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad9199-1a2e-49d7-991a-149ef1542148 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:16.528910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a4ad9199-1a2e-49d7-991a-149ef1542148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a8e9dddc3b841508f5878a72d22081e', 'request_id': 'req-a4ad9199-1a2e-49d7-991a-149ef1542148'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:16.538339 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a4ad9199-1a2e-49d7-991a-149ef1542148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a8e9dddc3b841508f5878a72d22081e', 'request_id': 'req-a4ad9199-1a2e-49d7-991a-149ef1542148'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:16.548458 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a4ad9199-1a2e-49d7-991a-149ef1542148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a8e9dddc3b841508f5878a72d22081e', 'request_id': 'req-a4ad9199-1a2e-49d7-991a-149ef1542148'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:16.565792 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a4ad9199-1a2e-49d7-991a-149ef1542148 admin admin] The token cache is being invalidate because project 1a8e9dddc3b841508f5878a72d22081e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:19:16.566098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a4ad9199-1a2e-49d7-991a-149ef1542148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:16.566812 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-a4ad9199-1a2e-49d7-991a-149ef1542148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:16.568421 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1200/2404] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:19:16 2022] DELETE /identity/v3/projects/1a8e9dddc3b841508f5878a72d22081e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:16.573789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a4d7c-3d16-445f-acbc-2031ef87f19b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:16.574370 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a4d7c-3d16-445f-acbc-2031ef87f19b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:16.574370 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a4d7c-3d16-445f-acbc-2031ef87f19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:16.594770 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a23a4d7c-3d16-445f-acbc-2031ef87f19b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:16.596281 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a23a4d7c-3d16-445f-acbc-2031ef87f19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:16.662746 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1205/2405] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:16.671238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916aac7a-30da-45b7-b3dd-9ba5f8821975 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:16.671554 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916aac7a-30da-45b7-b3dd-9ba5f8821975 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:16.671877 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916aac7a-30da-45b7-b3dd-9ba5f8821975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:16.694950 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-916aac7a-30da-45b7-b3dd-9ba5f8821975 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:16.696384 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-916aac7a-30da-45b7-b3dd-9ba5f8821975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:16.705592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1201/2406] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:16.709944 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a4d7c-3d16-445f-acbc-2031ef87f19b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:16.720501 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a4d7c-3d16-445f-acbc-2031ef87f19b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:16.726203 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d704b3-5150-4592-afc2-a1e77c5cdcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:16.726811 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d704b3-5150-4592-afc2-a1e77c5cdcbb admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:16.727178 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d704b3-5150-4592-afc2-a1e77c5cdcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:16.727455 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d704b3-5150-4592-afc2-a1e77c5cdcbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:16.728232 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d704b3-5150-4592-afc2-a1e77c5cdcbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:19:16.729467 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d704b3-5150-4592-afc2-a1e77c5cdcbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:16.731163 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d704b3-5150-4592-afc2-a1e77c5cdcbb admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:16.745302 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1206/2407] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:19:16.749829 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916aac7a-30da-45b7-b3dd-9ba5f8821975 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:16.760406 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916aac7a-30da-45b7-b3dd-9ba5f8821975 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:16.766100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dfea25-5782-460f-a0bc-9339280b06d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:16.766940 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfea25-5782-460f-a0bc-9339280b06d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:16.767753 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfea25-5782-460f-a0bc-9339280b06d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:16.767753 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfea25-5782-460f-a0bc-9339280b06d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:16.768525 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dfea25-5782-460f-a0bc-9339280b06d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:16.772324 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dfea25-5782-460f-a0bc-9339280b06d8 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:16.803675 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1202/2408] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:19:16 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:19:16.819908 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d704b3-5150-4592-afc2-a1e77c5cdcbb admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:16.839095 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d704b3-5150-4592-afc2-a1e77c5cdcbb admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:16.898089 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496322ce-e935-49d2-b7af-3ad85ba92f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:16.898634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496322ce-e935-49d2-b7af-3ad85ba92f9a admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:16.898959 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496322ce-e935-49d2-b7af-3ad85ba92f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:16.899241 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496322ce-e935-49d2-b7af-3ad85ba92f9a admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:16.899885 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496322ce-e935-49d2-b7af-3ad85ba92f9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:16.902377 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496322ce-e935-49d2-b7af-3ad85ba92f9a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:16.928220 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1207/2409] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:19:16 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:19:16.932857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dfea25-5782-460f-a0bc-9339280b06d8 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:16.944638 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dfea25-5782-460f-a0bc-9339280b06d8 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:16.949582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836bded-8bd7-4d0b-9bfa-b75c16ada5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:16.950194 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836bded-8bd7-4d0b-9bfa-b75c16ada5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:16.950478 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836bded-8bd7-4d0b-9bfa-b75c16ada5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:16.950756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836bded-8bd7-4d0b-9bfa-b75c16ada5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:16.951500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836bded-8bd7-4d0b-9bfa-b75c16ada5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:16.953181 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836bded-8bd7-4d0b-9bfa-b75c16ada5e1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:16.963975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1203/2410] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:16.968086 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496322ce-e935-49d2-b7af-3ad85ba92f9a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:16.977989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496322ce-e935-49d2-b7af-3ad85ba92f9a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:16.983351 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2931a-f490-4a26-ae3d-597bf94f42ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:16.983952 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2931a-f490-4a26-ae3d-597bf94f42ac admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:16.984235 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2931a-f490-4a26-ae3d-597bf94f42ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:16.984509 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2931a-f490-4a26-ae3d-597bf94f42ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:16.985151 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2931a-f490-4a26-ae3d-597bf94f42ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:16.986746 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2931a-f490-4a26-ae3d-597bf94f42ac admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:16.997968 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1208/2411] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:17.002017 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836bded-8bd7-4d0b-9bfa-b75c16ada5e1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:17.012935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836bded-8bd7-4d0b-9bfa-b75c16ada5e1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:17.017591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a86cc7-f9c4-4f12-8efb-028dc9a9e886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:17.018240 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a86cc7-f9c4-4f12-8efb-028dc9a9e886 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:17.018520 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a86cc7-f9c4-4f12-8efb-028dc9a9e886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:17.018793 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a86cc7-f9c4-4f12-8efb-028dc9a9e886 admin admin] PATH_INFO: `/v3/projects/f6e8b46d59984157af3cfd5f5b586023/users/12ab541374b34d0698c8189e019f07a9/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:17.034582 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a86cc7-f9c4-4f12-8efb-028dc9a9e886 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6e8b46d59984157af3cfd5f5b586023, user_id=12ab541374b34d0698c8189e019f07a9, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:17.037546 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a86cc7-f9c4-4f12-8efb-028dc9a9e886 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:17.050943 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1204/2412] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:19:17 2022] PUT /identity/v3/projects/f6e8b46d59984157af3cfd5f5b586023/users/12ab541374b34d0698c8189e019f07a9/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:17.062451 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe3661-40c2-47e5-a7bb-048381b89be6 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:17.062737 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe3661-40c2-47e5-a7bb-048381b89be6 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:17.063044 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe3661-40c2-47e5-a7bb-048381b89be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:17.094140 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-37fe3661-40c2-47e5-a7bb-048381b89be6 None None] MFA Rules not processed for user `12ab541374b34d0698c8189e019f07a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:17.095408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-37fe3661-40c2-47e5-a7bb-048381b89be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:17.142342 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:17.175489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1209/2413] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:17.180515 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a86cc7-f9c4-4f12-8efb-028dc9a9e886 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:17.251786 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a86cc7-f9c4-4f12-8efb-028dc9a9e886 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:17.257133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256f45d-02f1-4ee5-b97e-4306af881439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:17.257819 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256f45d-02f1-4ee5-b97e-4306af881439 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:17.258140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256f45d-02f1-4ee5-b97e-4306af881439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:17.258426 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256f45d-02f1-4ee5-b97e-4306af881439 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:17.259159 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256f45d-02f1-4ee5-b97e-4306af881439 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:17.261815 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256f45d-02f1-4ee5-b97e-4306af881439 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:17.286886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:17.296526 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1205/2414] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:19:17 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:19:17.300722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe3661-40c2-47e5-a7bb-048381b89be6 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:17.311658 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe3661-40c2-47e5-a7bb-048381b89be6 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:17.367752 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ca8c6-f8f9-42ef-89d7-0935757fce34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:17.368449 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ca8c6-f8f9-42ef-89d7-0935757fce34 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:17.368731 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ca8c6-f8f9-42ef-89d7-0935757fce34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:17.369004 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ca8c6-f8f9-42ef-89d7-0935757fce34 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:17.369827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ca8c6-f8f9-42ef-89d7-0935757fce34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:17.372047 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ca8c6-f8f9-42ef-89d7-0935757fce34 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:17.394846 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1210/2415] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:19:17 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:19:17.399299 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256f45d-02f1-4ee5-b97e-4306af881439 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:17.410144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256f45d-02f1-4ee5-b97e-4306af881439 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:17.415094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bfb60-b70c-49e2-9224-4607574a71c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:17.415707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bfb60-b70c-49e2-9224-4607574a71c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:17.417197 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bfb60-b70c-49e2-9224-4607574a71c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:17.417197 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bfb60-b70c-49e2-9224-4607574a71c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:17.417423 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113bfb60-b70c-49e2-9224-4607574a71c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:17.419040 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113bfb60-b70c-49e2-9224-4607574a71c1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:17.430283 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1206/2416] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:17.434572 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ca8c6-f8f9-42ef-89d7-0935757fce34 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:17.444428 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ca8c6-f8f9-42ef-89d7-0935757fce34 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:17.449798 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e3cae-7f81-499b-b056-226b352d16e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:17.450437 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e3cae-7f81-499b-b056-226b352d16e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:17.450715 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e3cae-7f81-499b-b056-226b352d16e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:17.451021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e3cae-7f81-499b-b056-226b352d16e4 admin admin] PATH_INFO: `/v3/projects/fd2058d31388445a8721f8e5250d0868/users/34f0b254964b40cab753276a6872f321/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:17.466562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e3cae-7f81-499b-b056-226b352d16e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd2058d31388445a8721f8e5250d0868, user_id=34f0b254964b40cab753276a6872f321, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:17.469218 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e3cae-7f81-499b-b056-226b352d16e4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:17.482003 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1211/2417] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:19:17 2022] PUT /identity/v3/projects/fd2058d31388445a8721f8e5250d0868/users/34f0b254964b40cab753276a6872f321/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:17.487474 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bfb60-b70c-49e2-9224-4607574a71c1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:17.497325 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bfb60-b70c-49e2-9224-4607574a71c1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:17.553664 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653dd03-6c20-4704-9f86-a80bccebdc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:17.554285 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653dd03-6c20-4704-9f86-a80bccebdc3f admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:17.554568 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653dd03-6c20-4704-9f86-a80bccebdc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:17.554843 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653dd03-6c20-4704-9f86-a80bccebdc3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:17.555519 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653dd03-6c20-4704-9f86-a80bccebdc3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:17.558004 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653dd03-6c20-4704-9f86-a80bccebdc3f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:17.568260 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1207/2418] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:17.572433 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e3cae-7f81-499b-b056-226b352d16e4 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:17.583099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e3cae-7f81-499b-b056-226b352d16e4 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:17.587607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db93f76-4f7d-4d05-a86a-f844609cfbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:17.588267 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db93f76-4f7d-4d05-a86a-f844609cfbfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:17.588712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db93f76-4f7d-4d05-a86a-f844609cfbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:17.589004 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db93f76-4f7d-4d05-a86a-f844609cfbfe admin admin] PATH_INFO: `/v3/projects/fd2058d31388445a8721f8e5250d0868/users/34f0b254964b40cab753276a6872f321/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:17.590911 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db93f76-4f7d-4d05-a86a-f844609cfbfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd2058d31388445a8721f8e5250d0868, user_id=34f0b254964b40cab753276a6872f321, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:17.593515 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db93f76-4f7d-4d05-a86a-f844609cfbfe admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:17.603406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1212/2419] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:19:17 2022] PUT /identity/v3/projects/fd2058d31388445a8721f8e5250d0868/users/34f0b254964b40cab753276a6872f321/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:17.613452 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93f41e-28d5-4c4d-b1ae-46942fcf84d2 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:17.613775 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93f41e-28d5-4c4d-b1ae-46942fcf84d2 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:17.614086 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93f41e-28d5-4c4d-b1ae-46942fcf84d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:17.643937 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-ac93f41e-28d5-4c4d-b1ae-46942fcf84d2 None None] MFA Rules not processed for user `34f0b254964b40cab753276a6872f321`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:17.645413 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ac93f41e-28d5-4c4d-b1ae-46942fcf84d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:17.720361 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1208/2420] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:17.731265 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db93f76-4f7d-4d05-a86a-f844609cfbfe admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:17.733051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-6db93f76-4f7d-4d05-a86a-f844609cfbfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:17.790501 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db93f76-4f7d-4d05-a86a-f844609cfbfe admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:17.794255 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda566d2-1be9-4ca0-b51f-f14348de8e6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:17.794879 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda566d2-1be9-4ca0-b51f-f14348de8e6d service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:17.795165 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda566d2-1be9-4ca0-b51f-f14348de8e6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:17.795461 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda566d2-1be9-4ca0-b51f-f14348de8e6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:17.841264 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:17.855055 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda566d2-1be9-4ca0-b51f-f14348de8e6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:17.856543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda566d2-1be9-4ca0-b51f-f14348de8e6d service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:17.868607 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1213/2421] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:19:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:19:23.554331 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93f41e-28d5-4c4d-b1ae-46942fcf84d2 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:23.555775 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-ac93f41e-28d5-4c4d-b1ae-46942fcf84d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:23.568094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93f41e-28d5-4c4d-b1ae-46942fcf84d2 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:23.571653 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62485b46-ef5c-4a3a-820a-d15090bfe3b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:23.572250 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62485b46-ef5c-4a3a-820a-d15090bfe3b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:23.573122 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62485b46-ef5c-4a3a-820a-d15090bfe3b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:23.573415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62485b46-ef5c-4a3a-820a-d15090bfe3b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:23.582058 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62485b46-ef5c-4a3a-820a-d15090bfe3b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:23.583508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62485b46-ef5c-4a3a-820a-d15090bfe3b5 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:23.592774 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1209/2422] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:19:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:19:24.522188 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda566d2-1be9-4ca0-b51f-f14348de8e6d service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:24.523664 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cda566d2-1be9-4ca0-b51f-f14348de8e6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:24.583818 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda566d2-1be9-4ca0-b51f-f14348de8e6d service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:24.587352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b34cd6-122b-42d2-a1b6-5d6a369d1eb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:24.587974 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b34cd6-122b-42d2-a1b6-5d6a369d1eb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:24.588317 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b34cd6-122b-42d2-a1b6-5d6a369d1eb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:24.588597 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b34cd6-122b-42d2-a1b6-5d6a369d1eb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:24.590328 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-27b34cd6-122b-42d2-a1b6-5d6a369d1eb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:24.651815 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b34cd6-122b-42d2-a1b6-5d6a369d1eb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:24.653254 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b34cd6-122b-42d2-a1b6-5d6a369d1eb5 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:24.662920 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1214/2423] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:19:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:19:33.588734 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=75 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:34.658374 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=57 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:50.656273 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62485b46-ef5c-4a3a-820a-d15090bfe3b5 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:50.657731 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-62485b46-ef5c-4a3a-820a-d15090bfe3b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:50.786457 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62485b46-ef5c-4a3a-820a-d15090bfe3b5 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:50.791699 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb4e29-2b55-4a04-aba2-5d9ec7ffa537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:50.792332 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb4e29-2b55-4a04-aba2-5d9ec7ffa537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:50.792612 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb4e29-2b55-4a04-aba2-5d9ec7ffa537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:50.792886 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb4e29-2b55-4a04-aba2-5d9ec7ffa537 admin admin] PATH_INFO: `/v3/users/5c4bcd13733e499ab0a6267675d5acd5` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:50.794005 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb4e29-2b55-4a04-aba2-5d9ec7ffa537 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c4bcd13733e499ab0a6267675d5acd5)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:50.795962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb4e29-2b55-4a04-aba2-5d9ec7ffa537 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:50.841931 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fccb4e29-2b55-4a04-aba2-5d9ec7ffa537 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c4bcd13733e499ab0a6267675d5acd5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:50.849811 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fccb4e29-2b55-4a04-aba2-5d9ec7ffa537 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c4bcd13733e499ab0a6267675d5acd5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:50.850464 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fccb4e29-2b55-4a04-aba2-5d9ec7ffa537 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c4bcd13733e499ab0a6267675d5acd5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:50.857296 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fccb4e29-2b55-4a04-aba2-5d9ec7ffa537 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c4bcd13733e499ab0a6267675d5acd5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:50.869182 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fccb4e29-2b55-4a04-aba2-5d9ec7ffa537 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c4bcd13733e499ab0a6267675d5acd5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:50.874922 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1210/2424] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:19:50 2022] DELETE /identity/v3/users/5c4bcd13733e499ab0a6267675d5acd5 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:50.879435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecdb4f-495c-46a8-9d9f-6c78acad729a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:50.879941 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecdb4f-495c-46a8-9d9f-6c78acad729a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:50.880046 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecdb4f-495c-46a8-9d9f-6c78acad729a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:50.907293 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-cbecdb4f-495c-46a8-9d9f-6c78acad729a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:50.908453 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cbecdb4f-495c-46a8-9d9f-6c78acad729a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:50.966413 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1215/2425] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:50.976459 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb4e29-2b55-4a04-aba2-5d9ec7ffa537 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:50.977856 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fccb4e29-2b55-4a04-aba2-5d9ec7ffa537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:50.999746 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:51.061909 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb4e29-2b55-4a04-aba2-5d9ec7ffa537 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:51.066743 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27a2b9-f48e-4f45-ac56-ff0861b83d8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:51.067795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27a2b9-f48e-4f45-ac56-ff0861b83d8e service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:51.067795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27a2b9-f48e-4f45-ac56-ff0861b83d8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:51.067795 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27a2b9-f48e-4f45-ac56-ff0861b83d8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:51.076935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27a2b9-f48e-4f45-ac56-ff0861b83d8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:51.078364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27a2b9-f48e-4f45-ac56-ff0861b83d8e service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:51.089083 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1211/2426] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:19:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:19:51.374671 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbecdb4f-495c-46a8-9d9f-6c78acad729a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:51.376167 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-cbecdb4f-495c-46a8-9d9f-6c78acad729a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:51.389592 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbecdb4f-495c-46a8-9d9f-6c78acad729a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:51.395971 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a64e9-3e6c-4de0-8ea5-ba4fa179eaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:51.396629 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a64e9-3e6c-4de0-8ea5-ba4fa179eaee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:51.397023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a64e9-3e6c-4de0-8ea5-ba4fa179eaee admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:51.397325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a64e9-3e6c-4de0-8ea5-ba4fa179eaee admin admin] PATH_INFO: `/v3/projects/73ec65cc06544a6d86bb4059de5d3d67` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:51.398576 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a64e9-3e6c-4de0-8ea5-ba4fa179eaee admin admin] RBAC: Authorizing `identity:delete_project(project_id=73ec65cc06544a6d86bb4059de5d3d67)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:51.400463 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a64e9-3e6c-4de0-8ea5-ba4fa179eaee admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:51.461710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9a8a64e9-3e6c-4de0-8ea5-ba4fa179eaee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73ec65cc06544a6d86bb4059de5d3d67', 'request_id': 'req-9a8a64e9-3e6c-4de0-8ea5-ba4fa179eaee'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:51.467344 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9a8a64e9-3e6c-4de0-8ea5-ba4fa179eaee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73ec65cc06544a6d86bb4059de5d3d67', 'request_id': 'req-9a8a64e9-3e6c-4de0-8ea5-ba4fa179eaee'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:51.474021 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9a8a64e9-3e6c-4de0-8ea5-ba4fa179eaee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73ec65cc06544a6d86bb4059de5d3d67', 'request_id': 'req-9a8a64e9-3e6c-4de0-8ea5-ba4fa179eaee'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:51.489300 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9a8a64e9-3e6c-4de0-8ea5-ba4fa179eaee admin admin] The token cache is being invalidate because project 73ec65cc06544a6d86bb4059de5d3d67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:19:51.489872 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9a8a64e9-3e6c-4de0-8ea5-ba4fa179eaee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:51.490538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-9a8a64e9-3e6c-4de0-8ea5-ba4fa179eaee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:51.492069 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1216/2427] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:19:51 2022] DELETE /identity/v3/projects/73ec65cc06544a6d86bb4059de5d3d67 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:51.497575 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bab2f-e8c0-4487-b2fa-2089ab11e51a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:51.497905 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bab2f-e8c0-4487-b2fa-2089ab11e51a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:51.498183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bab2f-e8c0-4487-b2fa-2089ab11e51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:51.549456 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-c29bab2f-e8c0-4487-b2fa-2089ab11e51a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:51.551218 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c29bab2f-e8c0-4487-b2fa-2089ab11e51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:51.612627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1212/2428] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:51.620503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7be03-05d1-4c6f-a501-9749625c3bea None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:51.620789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7be03-05d1-4c6f-a501-9749625c3bea None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:51.621067 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7be03-05d1-4c6f-a501-9749625c3bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:51.643665 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-55b7be03-05d1-4c6f-a501-9749625c3bea None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:51.644920 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-55b7be03-05d1-4c6f-a501-9749625c3bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:51.653159 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1217/2429] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:51.657633 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bab2f-e8c0-4487-b2fa-2089ab11e51a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:51.667428 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bab2f-e8c0-4487-b2fa-2089ab11e51a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:51.672739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39170bd-e6f0-4e2f-8621-3aa41c4b98bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:51.673316 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39170bd-e6f0-4e2f-8621-3aa41c4b98bf admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:51.673592 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39170bd-e6f0-4e2f-8621-3aa41c4b98bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:51.673858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39170bd-e6f0-4e2f-8621-3aa41c4b98bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:51.674514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39170bd-e6f0-4e2f-8621-3aa41c4b98bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:19:51.675898 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39170bd-e6f0-4e2f-8621-3aa41c4b98bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:51.677024 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39170bd-e6f0-4e2f-8621-3aa41c4b98bf admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:51.689834 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1213/2430] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:19:51.694793 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7be03-05d1-4c6f-a501-9749625c3bea None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:51.703716 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7be03-05d1-4c6f-a501-9749625c3bea None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:51.708759 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae52dda-c11b-4570-8726-08a8b441880e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:51.709387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae52dda-c11b-4570-8726-08a8b441880e admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:51.709668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae52dda-c11b-4570-8726-08a8b441880e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:51.709969 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae52dda-c11b-4570-8726-08a8b441880e admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:51.710569 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae52dda-c11b-4570-8726-08a8b441880e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:51.712848 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae52dda-c11b-4570-8726-08a8b441880e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:51.742632 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1218/2431] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:19:51 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:19:51.747100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39170bd-e6f0-4e2f-8621-3aa41c4b98bf admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:51.758419 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39170bd-e6f0-4e2f-8621-3aa41c4b98bf admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:51.816325 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcf4aa-1688-4912-b736-dd3a9f48f5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:51.816935 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcf4aa-1688-4912-b736-dd3a9f48f5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:51.817214 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcf4aa-1688-4912-b736-dd3a9f48f5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:51.817485 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcf4aa-1688-4912-b736-dd3a9f48f5a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:51.818134 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dcf4aa-1688-4912-b736-dd3a9f48f5a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:51.820060 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dcf4aa-1688-4912-b736-dd3a9f48f5a4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:51.839106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:51.850303 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1214/2432] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:19:51 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:19:51.851312 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae52dda-c11b-4570-8726-08a8b441880e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:51.862052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae52dda-c11b-4570-8726-08a8b441880e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:51.875606 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784705c-2473-435e-b80a-6b23c596b94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:51.876244 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784705c-2473-435e-b80a-6b23c596b94f admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:51.876522 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784705c-2473-435e-b80a-6b23c596b94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:51.876789 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784705c-2473-435e-b80a-6b23c596b94f admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:51.877440 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784705c-2473-435e-b80a-6b23c596b94f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:51.879234 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784705c-2473-435e-b80a-6b23c596b94f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:51.900728 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1219/2433] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:51.904866 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcf4aa-1688-4912-b736-dd3a9f48f5a4 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:51.915611 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcf4aa-1688-4912-b736-dd3a9f48f5a4 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:51.920242 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a22fe7-746c-48f3-9a91-f355b396c5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:51.920839 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a22fe7-746c-48f3-9a91-f355b396c5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:51.921124 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a22fe7-746c-48f3-9a91-f355b396c5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:51.921391 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a22fe7-746c-48f3-9a91-f355b396c5f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:51.922032 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a22fe7-746c-48f3-9a91-f355b396c5f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:51.923722 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a22fe7-746c-48f3-9a91-f355b396c5f7 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:51.935675 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1215/2434] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:51.941004 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784705c-2473-435e-b80a-6b23c596b94f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:51.951892 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784705c-2473-435e-b80a-6b23c596b94f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:51.957149 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed58a8-7312-4fa7-b975-54eb191c4e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:51.957798 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed58a8-7312-4fa7-b975-54eb191c4e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:51.958082 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed58a8-7312-4fa7-b975-54eb191c4e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:51.958389 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed58a8-7312-4fa7-b975-54eb191c4e1f admin admin] PATH_INFO: `/v3/projects/917f1029c33f4607b398287532f2fb4b/users/4d30a0379df643ff81e1ca01f3e26d3d/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:51.974230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed58a8-7312-4fa7-b975-54eb191c4e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=917f1029c33f4607b398287532f2fb4b, user_id=4d30a0379df643ff81e1ca01f3e26d3d, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:51.977002 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed58a8-7312-4fa7-b975-54eb191c4e1f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:51.989734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1220/2435] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:19:51 2022] PUT /identity/v3/projects/917f1029c33f4607b398287532f2fb4b/users/4d30a0379df643ff81e1ca01f3e26d3d/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:51.995009 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d89521-1564-454c-858f-a545d4946d4a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:51.995297 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d89521-1564-454c-858f-a545d4946d4a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:51.995572 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d89521-1564-454c-858f-a545d4946d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:52.017314 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-64d89521-1564-454c-858f-a545d4946d4a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:52.019548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-64d89521-1564-454c-858f-a545d4946d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:52.081804 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1216/2436] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:52.091631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed58a8-7312-4fa7-b975-54eb191c4e1f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:52.093444 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-14ed58a8-7312-4fa7-b975-54eb191c4e1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:52.107754 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:52.154123 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed58a8-7312-4fa7-b975-54eb191c4e1f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:52.158489 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d5814-0b33-42ca-8f67-be89bb13b982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:52.158632 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d5814-0b33-42ca-8f67-be89bb13b982 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:52.158820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d5814-0b33-42ca-8f67-be89bb13b982 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:52.159725 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d5814-0b33-42ca-8f67-be89bb13b982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:52.169501 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d5814-0b33-42ca-8f67-be89bb13b982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:52.170926 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d5814-0b33-42ca-8f67-be89bb13b982 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:52.181163 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1221/2437] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:19:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:19:55.822170 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d89521-1564-454c-858f-a545d4946d4a None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:55.823651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-64d89521-1564-454c-858f-a545d4946d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:55.837382 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d89521-1564-454c-858f-a545d4946d4a None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:55.843498 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cc661-483f-4933-8386-3f1e2aeef830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:55.844466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cc661-483f-4933-8386-3f1e2aeef830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:55.844837 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cc661-483f-4933-8386-3f1e2aeef830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:55.845209 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cc661-483f-4933-8386-3f1e2aeef830 admin admin] PATH_INFO: `/v3/users/12ab541374b34d0698c8189e019f07a9` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:55.846400 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cc661-483f-4933-8386-3f1e2aeef830 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12ab541374b34d0698c8189e019f07a9)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:55.848500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7cc661-483f-4933-8386-3f1e2aeef830 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:55.948662 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cb7cc661-483f-4933-8386-3f1e2aeef830 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12ab541374b34d0698c8189e019f07a9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:55.964862 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cb7cc661-483f-4933-8386-3f1e2aeef830 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12ab541374b34d0698c8189e019f07a9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:55.966164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cb7cc661-483f-4933-8386-3f1e2aeef830 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12ab541374b34d0698c8189e019f07a9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:55.988827 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cb7cc661-483f-4933-8386-3f1e2aeef830 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12ab541374b34d0698c8189e019f07a9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:56.012132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-cb7cc661-483f-4933-8386-3f1e2aeef830 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12ab541374b34d0698c8189e019f07a9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:56.023105 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1217/2438] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:19:55 2022] DELETE /identity/v3/users/12ab541374b34d0698c8189e019f07a9 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:19:56.027564 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033b143-a279-4d0f-8469-7b3b12734f73 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:56.028101 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033b143-a279-4d0f-8469-7b3b12734f73 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:56.028188 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033b143-a279-4d0f-8469-7b3b12734f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:56.056996 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8033b143-a279-4d0f-8469-7b3b12734f73 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:56.059781 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8033b143-a279-4d0f-8469-7b3b12734f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:56.146198 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1222/2439] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:19:56.155777 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cc661-483f-4933-8386-3f1e2aeef830 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:56.157246 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-cb7cc661-483f-4933-8386-3f1e2aeef830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:56.236323 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a23122-a650-4108-8a1a-923f2878a085 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:56.236613 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a23122-a650-4108-8a1a-923f2878a085 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:56.237001 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a23122-a650-4108-8a1a-923f2878a085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:56.274690 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cc661-483f-4933-8386-3f1e2aeef830 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:56.274690 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9903047-6b86-4276-aef6-1194d4b5b7a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:56.274690 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9903047-6b86-4276-aef6-1194d4b5b7a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:56.274690 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9903047-6b86-4276-aef6-1194d4b5b7a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:56.275630 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9903047-6b86-4276-aef6-1194d4b5b7a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:56.290737 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-62a23122-a650-4108-8a1a-923f2878a085 None None] MFA Rules not processed for user `4d30a0379df643ff81e1ca01f3e26d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:56.292140 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-62a23122-a650-4108-8a1a-923f2878a085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:56.294166 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9903047-6b86-4276-aef6-1194d4b5b7a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:56.295730 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9903047-6b86-4276-aef6-1194d4b5b7a3 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:56.316276 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1218/2440] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:19:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:19:56.432187 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1223/2441] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:56.447446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9903047-6b86-4276-aef6-1194d4b5b7a3 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:56.449849 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-f9903047-6b86-4276-aef6-1194d4b5b7a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:56.466064 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:56.482777 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9903047-6b86-4276-aef6-1194d4b5b7a3 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:56.491677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596e837-81fa-4095-b1b8-b0a507f465c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:56.492926 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596e837-81fa-4095-b1b8-b0a507f465c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:56.493272 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596e837-81fa-4095-b1b8-b0a507f465c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:56.493556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596e837-81fa-4095-b1b8-b0a507f465c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:56.494293 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596e837-81fa-4095-b1b8-b0a507f465c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:56.497171 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596e837-81fa-4095-b1b8-b0a507f465c1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:56.546443 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1219/2442] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:19:56 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:19:56.550712 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a23122-a650-4108-8a1a-923f2878a085 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:56.573792 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a23122-a650-4108-8a1a-923f2878a085 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:56.661162 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=100 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:56.675815 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2ae6b-d41d-48a3-b99f-6c011fc10df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:56.676716 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2ae6b-d41d-48a3-b99f-6c011fc10df1 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:56.677084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2ae6b-d41d-48a3-b99f-6c011fc10df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:56.677458 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2ae6b-d41d-48a3-b99f-6c011fc10df1 admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:56.678192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2ae6b-d41d-48a3-b99f-6c011fc10df1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:56.680427 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2ae6b-d41d-48a3-b99f-6c011fc10df1 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:56.710327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1224/2443] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:19:56 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:19:56.714345 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596e837-81fa-4095-b1b8-b0a507f465c1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:56.727387 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596e837-81fa-4095-b1b8-b0a507f465c1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:56.735284 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71c5dd-e3c3-4118-a7b2-d93cebd2b8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:56.736409 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71c5dd-e3c3-4118-a7b2-d93cebd2b8cd admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:56.737751 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71c5dd-e3c3-4118-a7b2-d93cebd2b8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:56.738588 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71c5dd-e3c3-4118-a7b2-d93cebd2b8cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:56.739506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71c5dd-e3c3-4118-a7b2-d93cebd2b8cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:56.741759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71c5dd-e3c3-4118-a7b2-d93cebd2b8cd admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:56.759553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1220/2444] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:56.764186 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2ae6b-d41d-48a3-b99f-6c011fc10df1 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:56.776661 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2ae6b-d41d-48a3-b99f-6c011fc10df1 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:56.782617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f984ff3-4286-4cf2-977d-68a5d1bdb130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:56.783428 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f984ff3-4286-4cf2-977d-68a5d1bdb130 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:56.784484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f984ff3-4286-4cf2-977d-68a5d1bdb130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:56.784868 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f984ff3-4286-4cf2-977d-68a5d1bdb130 admin admin] PATH_INFO: `/v3/projects/bd52b177a16a4ea1b2dff23d32ab4673/users/b86d602523364d58bcd0d5df3aadb8eb/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:56.803368 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f984ff3-4286-4cf2-977d-68a5d1bdb130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd52b177a16a4ea1b2dff23d32ab4673, user_id=b86d602523364d58bcd0d5df3aadb8eb, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:56.806631 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f984ff3-4286-4cf2-977d-68a5d1bdb130 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:56.823454 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1225/2445] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:19:56 2022] PUT /identity/v3/projects/bd52b177a16a4ea1b2dff23d32ab4673/users/b86d602523364d58bcd0d5df3aadb8eb/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:56.827416 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71c5dd-e3c3-4118-a7b2-d93cebd2b8cd admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:56.838506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71c5dd-e3c3-4118-a7b2-d93cebd2b8cd admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:56.848859 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f984ff3-4286-4cf2-977d-68a5d1bdb130 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:56.850652 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8f984ff3-4286-4cf2-977d-68a5d1bdb130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:56.905711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037179d-0824-4e35-aff7-c20fff921f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:56.906327 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037179d-0824-4e35-aff7-c20fff921f0e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:56.906606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037179d-0824-4e35-aff7-c20fff921f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:56.906891 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037179d-0824-4e35-aff7-c20fff921f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:56.907550 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037179d-0824-4e35-aff7-c20fff921f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:56.909317 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037179d-0824-4e35-aff7-c20fff921f0e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:56.917586 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f984ff3-4286-4cf2-977d-68a5d1bdb130 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:56.919831 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1221/2446] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:56.922526 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e442601-915a-48e1-822b-6a1f6bdb231a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:56.923192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e442601-915a-48e1-822b-6a1f6bdb231a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:56.923477 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e442601-915a-48e1-822b-6a1f6bdb231a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:56.923751 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e442601-915a-48e1-822b-6a1f6bdb231a admin admin] PATH_INFO: `/v3/projects/f6e8b46d59984157af3cfd5f5b586023` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:56.923973 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037179d-0824-4e35-aff7-c20fff921f0e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:56.925417 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e442601-915a-48e1-822b-6a1f6bdb231a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6e8b46d59984157af3cfd5f5b586023)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:56.927373 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e442601-915a-48e1-822b-6a1f6bdb231a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:56.933986 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037179d-0824-4e35-aff7-c20fff921f0e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:56.939841 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839e20d-ddad-422d-8f6a-7ffa5d6b033e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:56.941258 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839e20d-ddad-422d-8f6a-7ffa5d6b033e admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:56.941694 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839e20d-ddad-422d-8f6a-7ffa5d6b033e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:56.941977 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839e20d-ddad-422d-8f6a-7ffa5d6b033e admin admin] PATH_INFO: `/v3/projects/bd52b177a16a4ea1b2dff23d32ab4673/users/b86d602523364d58bcd0d5df3aadb8eb/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:56.944059 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839e20d-ddad-422d-8f6a-7ffa5d6b033e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd52b177a16a4ea1b2dff23d32ab4673, user_id=b86d602523364d58bcd0d5df3aadb8eb, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:56.946791 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839e20d-ddad-422d-8f6a-7ffa5d6b033e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:56.957645 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1222/2447] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:19:56 2022] PUT /identity/v3/projects/bd52b177a16a4ea1b2dff23d32ab4673/users/b86d602523364d58bcd0d5df3aadb8eb/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:19:56.977329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e442601-915a-48e1-822b-6a1f6bdb231a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6e8b46d59984157af3cfd5f5b586023', 'request_id': 'req-1e442601-915a-48e1-822b-6a1f6bdb231a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:56.982883 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e442601-915a-48e1-822b-6a1f6bdb231a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6e8b46d59984157af3cfd5f5b586023', 'request_id': 'req-1e442601-915a-48e1-822b-6a1f6bdb231a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:56.991289 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e442601-915a-48e1-822b-6a1f6bdb231a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6e8b46d59984157af3cfd5f5b586023', 'request_id': 'req-1e442601-915a-48e1-822b-6a1f6bdb231a'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:57.005226 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e442601-915a-48e1-822b-6a1f6bdb231a admin admin] The token cache is being invalidate because project f6e8b46d59984157af3cfd5f5b586023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:19:57.005696 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e442601-915a-48e1-822b-6a1f6bdb231a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:57.006591 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-1e442601-915a-48e1-822b-6a1f6bdb231a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:57.008371 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1226/2448] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:19:56 2022] DELETE /identity/v3/projects/f6e8b46d59984157af3cfd5f5b586023 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:19:57.013013 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839e20d-ddad-422d-8f6a-7ffa5d6b033e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:57.015650 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-3839e20d-ddad-422d-8f6a-7ffa5d6b033e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:57.090032 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839e20d-ddad-422d-8f6a-7ffa5d6b033e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:57.155334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f22e2e-3669-4690-acb1-239340debe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:57.155990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f22e2e-3669-4690-acb1-239340debe38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:57.156274 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f22e2e-3669-4690-acb1-239340debe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:57.156548 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f22e2e-3669-4690-acb1-239340debe38 admin admin] PATH_INFO: `/v3/users/34f0b254964b40cab753276a6872f321` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:57.157694 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f22e2e-3669-4690-acb1-239340debe38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34f0b254964b40cab753276a6872f321)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:57.160285 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f22e2e-3669-4690-acb1-239340debe38 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:57.161807 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:57.212168 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d2f22e2e-3669-4690-acb1-239340debe38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34f0b254964b40cab753276a6872f321'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:57.225389 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d2f22e2e-3669-4690-acb1-239340debe38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34f0b254964b40cab753276a6872f321'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:57.226507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d2f22e2e-3669-4690-acb1-239340debe38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34f0b254964b40cab753276a6872f321'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:57.235561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d2f22e2e-3669-4690-acb1-239340debe38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34f0b254964b40cab753276a6872f321'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:57.250607 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-d2f22e2e-3669-4690-acb1-239340debe38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34f0b254964b40cab753276a6872f321'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:57.260050 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1223/2449] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:19:57 2022] DELETE /identity/v3/users/34f0b254964b40cab753276a6872f321 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:57.599140 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e442601-915a-48e1-822b-6a1f6bdb231a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:57.617181 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e442601-915a-48e1-822b-6a1f6bdb231a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:57.697896 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:57.736327 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b21eb8-9f62-4c42-b1ef-bec8a54de1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:57.738028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b21eb8-9f62-4c42-b1ef-bec8a54de1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:57.738598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b21eb8-9f62-4c42-b1ef-bec8a54de1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:57.739032 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b21eb8-9f62-4c42-b1ef-bec8a54de1e4 admin admin] PATH_INFO: `/v3/projects/fd2058d31388445a8721f8e5250d0868` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:57.740589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b21eb8-9f62-4c42-b1ef-bec8a54de1e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd2058d31388445a8721f8e5250d0868)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:57.742776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b21eb8-9f62-4c42-b1ef-bec8a54de1e4 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:57.813723 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-53b21eb8-9f62-4c42-b1ef-bec8a54de1e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd2058d31388445a8721f8e5250d0868', 'request_id': 'req-53b21eb8-9f62-4c42-b1ef-bec8a54de1e4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:57.824897 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-53b21eb8-9f62-4c42-b1ef-bec8a54de1e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd2058d31388445a8721f8e5250d0868', 'request_id': 'req-53b21eb8-9f62-4c42-b1ef-bec8a54de1e4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:57.833854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-53b21eb8-9f62-4c42-b1ef-bec8a54de1e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd2058d31388445a8721f8e5250d0868', 'request_id': 'req-53b21eb8-9f62-4c42-b1ef-bec8a54de1e4'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:57.859010 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-53b21eb8-9f62-4c42-b1ef-bec8a54de1e4 admin admin] The token cache is being invalidate because project fd2058d31388445a8721f8e5250d0868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:19:57.859461 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-53b21eb8-9f62-4c42-b1ef-bec8a54de1e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:57.860171 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-53b21eb8-9f62-4c42-b1ef-bec8a54de1e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:19:57.862799 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1227/2450] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:19:57 2022] DELETE /identity/v3/projects/fd2058d31388445a8721f8e5250d0868 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:19:57.867530 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e84e65-f632-4759-b3ad-8927d70ee56d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:57.868046 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e84e65-f632-4759-b3ad-8927d70ee56d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:57.868142 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e84e65-f632-4759-b3ad-8927d70ee56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:57.892809 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-24e84e65-f632-4759-b3ad-8927d70ee56d None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:57.894613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-24e84e65-f632-4759-b3ad-8927d70ee56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:57.980364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1224/2451] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:57.990098 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b24ba-273e-4a8c-bbea-e667f6f62441 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:57.990098 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b24ba-273e-4a8c-bbea-e667f6f62441 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:57.990403 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b24ba-273e-4a8c-bbea-e667f6f62441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:58.014193 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3a8b24ba-273e-4a8c-bbea-e667f6f62441 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:58.015724 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3a8b24ba-273e-4a8c-bbea-e667f6f62441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:58.027005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1228/2452] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:19:58.030606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e84e65-f632-4759-b3ad-8927d70ee56d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:58.041309 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e84e65-f632-4759-b3ad-8927d70ee56d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:58.047759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed2e93-cf6a-45ba-b733-d6e57561dd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:58.048379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed2e93-cf6a-45ba-b733-d6e57561dd0b admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:58.048669 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed2e93-cf6a-45ba-b733-d6e57561dd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:58.048927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed2e93-cf6a-45ba-b733-d6e57561dd0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:58.049776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed2e93-cf6a-45ba-b733-d6e57561dd0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:19:58.050271 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed2e93-cf6a-45ba-b733-d6e57561dd0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:58.052063 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed2e93-cf6a-45ba-b733-d6e57561dd0b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:58.064612 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1225/2453] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:19:58.069743 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b24ba-273e-4a8c-bbea-e667f6f62441 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:58.080344 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b24ba-273e-4a8c-bbea-e667f6f62441 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:58.085870 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40073858-347a-4c9a-840f-d39655c24ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:58.086493 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40073858-347a-4c9a-840f-d39655c24ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:58.086778 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40073858-347a-4c9a-840f-d39655c24ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:58.087078 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40073858-347a-4c9a-840f-d39655c24ca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:58.087733 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40073858-347a-4c9a-840f-d39655c24ca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:58.089807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40073858-347a-4c9a-840f-d39655c24ca9 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:58.128816 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1229/2454] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:19:58 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:19:58.129496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed2e93-cf6a-45ba-b733-d6e57561dd0b admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:58.141958 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed2e93-cf6a-45ba-b733-d6e57561dd0b admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:58.218590 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a40e9d-a38d-41d7-a18f-7809b1f33ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:58.219115 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a40e9d-a38d-41d7-a18f-7809b1f33ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:58.219476 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a40e9d-a38d-41d7-a18f-7809b1f33ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:58.219821 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a40e9d-a38d-41d7-a18f-7809b1f33ca9 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:58.220546 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a40e9d-a38d-41d7-a18f-7809b1f33ca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:58.223204 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a40e9d-a38d-41d7-a18f-7809b1f33ca9 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:58.236550 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:58.257514 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40073858-347a-4c9a-840f-d39655c24ca9 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:58.258628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1226/2455] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:19:58 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:19:58.272799 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40073858-347a-4c9a-840f-d39655c24ca9 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:58.279798 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401390e-10e1-46b5-b606-280e79565afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:58.280713 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401390e-10e1-46b5-b606-280e79565afe admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:58.281151 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401390e-10e1-46b5-b606-280e79565afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:58.283664 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f401390e-10e1-46b5-b606-280e79565afe admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:58.285279 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401390e-10e1-46b5-b606-280e79565afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:58.287445 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f401390e-10e1-46b5-b606-280e79565afe admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:58.303392 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1230/2456] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:19:58.307203 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a40e9d-a38d-41d7-a18f-7809b1f33ca9 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:58.318359 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a40e9d-a38d-41d7-a18f-7809b1f33ca9 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:58.323485 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e2669-1537-4e38-ba03-c04a12f1b8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:58.324198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e2669-1537-4e38-ba03-c04a12f1b8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:58.324551 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e2669-1537-4e38-ba03-c04a12f1b8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:58.324895 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e2669-1537-4e38-ba03-c04a12f1b8b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:58.325611 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e2669-1537-4e38-ba03-c04a12f1b8b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:58.327341 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e2669-1537-4e38-ba03-c04a12f1b8b3 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:58.342469 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1227/2457] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:58.347425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401390e-10e1-46b5-b606-280e79565afe admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:58.358883 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f401390e-10e1-46b5-b606-280e79565afe admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:58.365964 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0107b-e566-47e6-9f55-e1ed6ce67523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:58.366806 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0107b-e566-47e6-9f55-e1ed6ce67523 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:58.367199 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0107b-e566-47e6-9f55-e1ed6ce67523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:58.367547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0107b-e566-47e6-9f55-e1ed6ce67523 admin admin] PATH_INFO: `/v3/projects/91c56fdfcf1948f9aab685ce5b1b9cbe/users/b114171a48d04ba1b21d8a1e19ee8463/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:58.389686 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0107b-e566-47e6-9f55-e1ed6ce67523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91c56fdfcf1948f9aab685ce5b1b9cbe, user_id=b114171a48d04ba1b21d8a1e19ee8463, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:58.392866 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0107b-e566-47e6-9f55-e1ed6ce67523 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:58.410213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1231/2458] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:19:58 2022] PUT /identity/v3/projects/91c56fdfcf1948f9aab685ce5b1b9cbe/users/b114171a48d04ba1b21d8a1e19ee8463/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:58.420804 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1152b7-838a-4ac1-bcc5-2d28ed2fccdc None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:58.421087 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1152b7-838a-4ac1-bcc5-2d28ed2fccdc None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:58.421377 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1152b7-838a-4ac1-bcc5-2d28ed2fccdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:58.467726 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2a1152b7-838a-4ac1-bcc5-2d28ed2fccdc None None] MFA Rules not processed for user `b114171a48d04ba1b21d8a1e19ee8463`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:58.469119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2a1152b7-838a-4ac1-bcc5-2d28ed2fccdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:58.555556 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1228/2459] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:58.557039 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0107b-e566-47e6-9f55-e1ed6ce67523 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:58.626042 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0107b-e566-47e6-9f55-e1ed6ce67523 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:58.631279 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52e78a-44d0-40cb-85ba-a375aee447f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:58.631894 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52e78a-44d0-40cb-85ba-a375aee447f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:58.632538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52e78a-44d0-40cb-85ba-a375aee447f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:58.632538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52e78a-44d0-40cb-85ba-a375aee447f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:58.633065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52e78a-44d0-40cb-85ba-a375aee447f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:58.635390 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52e78a-44d0-40cb-85ba-a375aee447f2 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:58.650136 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:58.670522 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1232/2460] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:19:58 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:19:58.672610 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1152b7-838a-4ac1-bcc5-2d28ed2fccdc None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:58.684469 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1152b7-838a-4ac1-bcc5-2d28ed2fccdc None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:58.750608 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043c7be-0910-4d48-9eed-963565a9f349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:58.751571 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043c7be-0910-4d48-9eed-963565a9f349 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:58.752005 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043c7be-0910-4d48-9eed-963565a9f349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:58.752454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043c7be-0910-4d48-9eed-963565a9f349 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:58.753449 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043c7be-0910-4d48-9eed-963565a9f349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:58.755765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043c7be-0910-4d48-9eed-963565a9f349 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:58.790513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1229/2461] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:19:58 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:19:58.795768 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52e78a-44d0-40cb-85ba-a375aee447f2 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:58.810915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52e78a-44d0-40cb-85ba-a375aee447f2 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:58.817331 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a367a4c-315b-47d9-a7e3-766d86857e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:58.818158 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a367a4c-315b-47d9-a7e3-766d86857e71 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:58.818450 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a367a4c-315b-47d9-a7e3-766d86857e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:58.818722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a367a4c-315b-47d9-a7e3-766d86857e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:58.819781 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a367a4c-315b-47d9-a7e3-766d86857e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:58.821538 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a367a4c-315b-47d9-a7e3-766d86857e71 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:58.838911 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1233/2462] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:58.840683 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043c7be-0910-4d48-9eed-963565a9f349 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:58.853665 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043c7be-0910-4d48-9eed-963565a9f349 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:58.860224 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36321dc-d4c8-49be-b6e5-b570a467189e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:58.860505 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36321dc-d4c8-49be-b6e5-b570a467189e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:58.860783 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36321dc-d4c8-49be-b6e5-b570a467189e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:58.862954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36321dc-d4c8-49be-b6e5-b570a467189e admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:58.863611 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36321dc-d4c8-49be-b6e5-b570a467189e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:58.865454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36321dc-d4c8-49be-b6e5-b570a467189e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:58.880692 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1230/2463] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:58.884980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a367a4c-315b-47d9-a7e3-766d86857e71 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:58.897816 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a367a4c-315b-47d9-a7e3-766d86857e71 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:58.909095 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8841740-a204-40d8-9d2f-0b03eb247f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:58.909771 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8841740-a204-40d8-9d2f-0b03eb247f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:58.910047 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8841740-a204-40d8-9d2f-0b03eb247f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:58.910319 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8841740-a204-40d8-9d2f-0b03eb247f74 admin admin] PATH_INFO: `/v3/projects/c41cf62e5be946c5ad61a51dd0bf90f1/users/60ef1c2f23144fa1a24ed824420ea3d3/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:58.939114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8841740-a204-40d8-9d2f-0b03eb247f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c41cf62e5be946c5ad61a51dd0bf90f1, user_id=60ef1c2f23144fa1a24ed824420ea3d3, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:58.943384 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8841740-a204-40d8-9d2f-0b03eb247f74 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:58.962283 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1234/2464] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:19:58 2022] PUT /identity/v3/projects/c41cf62e5be946c5ad61a51dd0bf90f1/users/60ef1c2f23144fa1a24ed824420ea3d3/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:19:58.974296 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01a9ca-1745-4f69-9124-3b0d99998c22 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:58.974296 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01a9ca-1745-4f69-9124-3b0d99998c22 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:58.974600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01a9ca-1745-4f69-9124-3b0d99998c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:59.007917 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-2e01a9ca-1745-4f69-9124-3b0d99998c22 None None] MFA Rules not processed for user `60ef1c2f23144fa1a24ed824420ea3d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:59.009205 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2e01a9ca-1745-4f69-9124-3b0d99998c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:59.065432 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:59.099406 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1231/2465] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:59.104282 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8841740-a204-40d8-9d2f-0b03eb247f74 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:59.192843 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8841740-a204-40d8-9d2f-0b03eb247f74 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:59.198147 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbbe87-d869-4f43-a550-8a821c349d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:59.198745 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbbe87-d869-4f43-a550-8a821c349d3d admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:59.199088 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbbe87-d869-4f43-a550-8a821c349d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:59.199374 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbbe87-d869-4f43-a550-8a821c349d3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:59.200071 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbbe87-d869-4f43-a550-8a821c349d3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:59.202183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbbe87-d869-4f43-a550-8a821c349d3d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:59.238733 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1235/2466] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:19:59 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:19:59.242979 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01a9ca-1745-4f69-9124-3b0d99998c22 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:59.255506 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01a9ca-1745-4f69-9124-3b0d99998c22 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:59.332927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499b645-b3a2-41b6-802c-24715bedffb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:59.333682 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499b645-b3a2-41b6-802c-24715bedffb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:59.333977 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499b645-b3a2-41b6-802c-24715bedffb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:59.334256 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499b645-b3a2-41b6-802c-24715bedffb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:59.334990 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499b645-b3a2-41b6-802c-24715bedffb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:59.336952 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499b645-b3a2-41b6-802c-24715bedffb3 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:59.365006 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1232/2467] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:19:59 2022] POST /identity/v3/users => generated 296 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:19:59.369556 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbbe87-d869-4f43-a550-8a821c349d3d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:59.388870 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbbe87-d869-4f43-a550-8a821c349d3d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:59.421561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ccd07-2c18-4cd2-94e9-d3b866864c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:59.422341 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ccd07-2c18-4cd2-94e9-d3b866864c49 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:59.423196 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ccd07-2c18-4cd2-94e9-d3b866864c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:59.423476 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ccd07-2c18-4cd2-94e9-d3b866864c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:59.424143 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ccd07-2c18-4cd2-94e9-d3b866864c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:59.426329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ccd07-2c18-4cd2-94e9-d3b866864c49 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:59.444662 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1236/2468] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:19:59.446009 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499b645-b3a2-41b6-802c-24715bedffb3 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:59.465527 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499b645-b3a2-41b6-802c-24715bedffb3 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:59.473164 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e561c-1cfe-4837-b383-784c5d9afc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:59.474799 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e561c-1cfe-4837-b383-784c5d9afc3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:59.475271 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e561c-1cfe-4837-b383-784c5d9afc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:59.475729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e561c-1cfe-4837-b383-784c5d9afc3f admin admin] PATH_INFO: `/v3/projects/0ad6546fe22540dcb26c81395481ec65/users/dae6a7ed1d2b4423bad1237a8976be84/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:59.509311 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e561c-1cfe-4837-b383-784c5d9afc3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad6546fe22540dcb26c81395481ec65, user_id=dae6a7ed1d2b4423bad1237a8976be84, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:59.513761 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e561c-1cfe-4837-b383-784c5d9afc3f admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:59.540179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1233/2469] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:19:59 2022] PUT /identity/v3/projects/0ad6546fe22540dcb26c81395481ec65/users/dae6a7ed1d2b4423bad1237a8976be84/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:19:59.544376 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ccd07-2c18-4cd2-94e9-d3b866864c49 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:59.557394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ccd07-2c18-4cd2-94e9-d3b866864c49 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:59.675899 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae705ca6-fdc9-410a-b836-410ffe200362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:59.676776 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae705ca6-fdc9-410a-b836-410ffe200362 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:59.677156 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae705ca6-fdc9-410a-b836-410ffe200362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:59.677536 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae705ca6-fdc9-410a-b836-410ffe200362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:59.678496 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae705ca6-fdc9-410a-b836-410ffe200362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:59.680587 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae705ca6-fdc9-410a-b836-410ffe200362 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:59.693675 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1237/2470] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:19:59.697957 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e561c-1cfe-4837-b383-784c5d9afc3f admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:59.713634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e561c-1cfe-4837-b383-784c5d9afc3f admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:19:59.720828 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d050b83-6a50-4d8a-8a2b-dd8b30b7a8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:19:59.721854 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d050b83-6a50-4d8a-8a2b-dd8b30b7a8e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:59.722238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d050b83-6a50-4d8a-8a2b-dd8b30b7a8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:59.722613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d050b83-6a50-4d8a-8a2b-dd8b30b7a8e0 admin admin] PATH_INFO: `/v3/projects/0ad6546fe22540dcb26c81395481ec65/users/dae6a7ed1d2b4423bad1237a8976be84/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:59.724893 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d050b83-6a50-4d8a-8a2b-dd8b30b7a8e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad6546fe22540dcb26c81395481ec65, user_id=dae6a7ed1d2b4423bad1237a8976be84, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:19:59.728174 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d050b83-6a50-4d8a-8a2b-dd8b30b7a8e0 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:19:59.753654 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1234/2471] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:19:59 2022] PUT /identity/v3/projects/0ad6546fe22540dcb26c81395481ec65/users/dae6a7ed1d2b4423bad1237a8976be84/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:19:59.814311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518acc0a-e955-4009-bfc2-64bf9d6a80d2 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:19:59.814602 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518acc0a-e955-4009-bfc2-64bf9d6a80d2 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:19:59.814896 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518acc0a-e955-4009-bfc2-64bf9d6a80d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:19:59.845547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-518acc0a-e955-4009-bfc2-64bf9d6a80d2 None None] MFA Rules not processed for user `dae6a7ed1d2b4423bad1237a8976be84`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:19:59.847570 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-518acc0a-e955-4009-bfc2-64bf9d6a80d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:19:59.868680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:19:59.935782 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1238/2472] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:19:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:19:59.949454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d050b83-6a50-4d8a-8a2b-dd8b30b7a8e0 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:19:59.952307 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9d050b83-6a50-4d8a-8a2b-dd8b30b7a8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:00.036172 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d050b83-6a50-4d8a-8a2b-dd8b30b7a8e0 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:00.043725 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeff857-76da-452b-aedd-d4f0279b2f9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:00.044805 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeff857-76da-452b-aedd-d4f0279b2f9b service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:00.045092 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeff857-76da-452b-aedd-d4f0279b2f9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:00.045365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeff857-76da-452b-aedd-d4f0279b2f9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:00.061679 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeff857-76da-452b-aedd-d4f0279b2f9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:00.063478 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeff857-76da-452b-aedd-d4f0279b2f9b service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:00.079252 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1235/2473] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:19:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:20:00.160941 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518acc0a-e955-4009-bfc2-64bf9d6a80d2 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:00.177100 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518acc0a-e955-4009-bfc2-64bf9d6a80d2 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:00.205286 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c191bb4-db5e-4c3a-a2d5-19bf4a4342b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:00.207274 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c191bb4-db5e-4c3a-a2d5-19bf4a4342b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:00.207562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c191bb4-db5e-4c3a-a2d5-19bf4a4342b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:00.207928 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c191bb4-db5e-4c3a-a2d5-19bf4a4342b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:00.356457 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c191bb4-db5e-4c3a-a2d5-19bf4a4342b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:00.359816 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c191bb4-db5e-4c3a-a2d5-19bf4a4342b5 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:00.394601 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1239/2474] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:20:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:20:01.253642 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeff857-76da-452b-aedd-d4f0279b2f9b service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:01.255270 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-7eeff857-76da-452b-aedd-d4f0279b2f9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:01.349642 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:01.414026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeff857-76da-452b-aedd-d4f0279b2f9b service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:01.426739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa966ec0-69ab-4842-a92e-3ed1059cc496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:01.429362 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa966ec0-69ab-4842-a92e-3ed1059cc496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:01.429680 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa966ec0-69ab-4842-a92e-3ed1059cc496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:01.431532 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa966ec0-69ab-4842-a92e-3ed1059cc496 admin admin] PATH_INFO: `/v3/users/d503e3ffa1be49d692deb0b38a37e5da` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:01.435127 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa966ec0-69ab-4842-a92e-3ed1059cc496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d503e3ffa1be49d692deb0b38a37e5da)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:01.442806 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa966ec0-69ab-4842-a92e-3ed1059cc496 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:01.554914 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fa966ec0-69ab-4842-a92e-3ed1059cc496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd503e3ffa1be49d692deb0b38a37e5da'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:01.569179 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fa966ec0-69ab-4842-a92e-3ed1059cc496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd503e3ffa1be49d692deb0b38a37e5da'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:01.570145 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fa966ec0-69ab-4842-a92e-3ed1059cc496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd503e3ffa1be49d692deb0b38a37e5da'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:01.585516 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fa966ec0-69ab-4842-a92e-3ed1059cc496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd503e3ffa1be49d692deb0b38a37e5da'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:01.608126 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-fa966ec0-69ab-4842-a92e-3ed1059cc496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd503e3ffa1be49d692deb0b38a37e5da'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:01.620966 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1236/2475] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:20:01 2022] DELETE /identity/v3/users/d503e3ffa1be49d692deb0b38a37e5da => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:20:01.625691 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c981e-d769-4aa4-8782-913e6d223b8f None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:01.626295 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c981e-d769-4aa4-8782-913e6d223b8f None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:01.626295 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c981e-d769-4aa4-8782-913e6d223b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:01.650299 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-206c981e-d769-4aa4-8782-913e6d223b8f None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:01.651718 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-206c981e-d769-4aa4-8782-913e6d223b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:01.772878 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1240/2476] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:20:01.784642 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa966ec0-69ab-4842-a92e-3ed1059cc496 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:01.786620 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fa966ec0-69ab-4842-a92e-3ed1059cc496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:01.928888 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa966ec0-69ab-4842-a92e-3ed1059cc496 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:01.936471 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f6bef-2982-42e7-9cd9-5f399e021c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:01.937113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f6bef-2982-42e7-9cd9-5f399e021c90 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:01.937401 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f6bef-2982-42e7-9cd9-5f399e021c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:01.939605 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f6bef-2982-42e7-9cd9-5f399e021c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:01.961422 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f6bef-2982-42e7-9cd9-5f399e021c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:01.964319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f6bef-2982-42e7-9cd9-5f399e021c90 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:01.987988 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1237/2477] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:20:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:20:02.475013 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c981e-d769-4aa4-8782-913e6d223b8f None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:02.476628 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-206c981e-d769-4aa4-8782-913e6d223b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:02.504617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c981e-d769-4aa4-8782-913e6d223b8f None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:02.513863 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2314d5-7b3f-42f8-a98c-a1fb779c36a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:02.514757 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2314d5-7b3f-42f8-a98c-a1fb779c36a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:02.515119 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2314d5-7b3f-42f8-a98c-a1fb779c36a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:02.515411 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2314d5-7b3f-42f8-a98c-a1fb779c36a5 admin admin] PATH_INFO: `/v3/projects/d3fa9697518749bfbe0b4297b824f4e5` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:02.517728 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2314d5-7b3f-42f8-a98c-a1fb779c36a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3fa9697518749bfbe0b4297b824f4e5)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:02.542049 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2314d5-7b3f-42f8-a98c-a1fb779c36a5 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:02.656681 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:02.660869 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4e2314d5-7b3f-42f8-a98c-a1fb779c36a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3fa9697518749bfbe0b4297b824f4e5', 'request_id': 'req-4e2314d5-7b3f-42f8-a98c-a1fb779c36a5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:02.668841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4e2314d5-7b3f-42f8-a98c-a1fb779c36a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3fa9697518749bfbe0b4297b824f4e5', 'request_id': 'req-4e2314d5-7b3f-42f8-a98c-a1fb779c36a5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:02.682146 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4e2314d5-7b3f-42f8-a98c-a1fb779c36a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3fa9697518749bfbe0b4297b824f4e5', 'request_id': 'req-4e2314d5-7b3f-42f8-a98c-a1fb779c36a5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:02.708260 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4e2314d5-7b3f-42f8-a98c-a1fb779c36a5 admin admin] The token cache is being invalidate because project d3fa9697518749bfbe0b4297b824f4e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:20:02.708564 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4e2314d5-7b3f-42f8-a98c-a1fb779c36a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:02.709621 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4e2314d5-7b3f-42f8-a98c-a1fb779c36a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:02.716568 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1241/2478] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:20:02 2022] DELETE /identity/v3/projects/d3fa9697518749bfbe0b4297b824f4e5 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:20:02.718231 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ae801-2223-4b79-9248-be05148a572a None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:02.719155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ae801-2223-4b79-9248-be05148a572a None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:02.719442 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ae801-2223-4b79-9248-be05148a572a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:02.735505 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88193b-bd0b-49d7-afbb-10322810a8c3 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:02.735825 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88193b-bd0b-49d7-afbb-10322810a8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:02.736039 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88193b-bd0b-49d7-afbb-10322810a8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:02.748197 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-441ae801-2223-4b79-9248-be05148a572a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:02.749700 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-441ae801-2223-4b79-9248-be05148a572a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:02.767983 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-3a88193b-bd0b-49d7-afbb-10322810a8c3 None None] MFA Rules not processed for user `b86d602523364d58bcd0d5df3aadb8eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:02.770381 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3a88193b-bd0b-49d7-afbb-10322810a8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:02.806639 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:02.858904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1238/2479] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:20:02.867074 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2ab36-eb72-497e-a090-2dc66dd16282 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:02.867296 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2ab36-eb72-497e-a090-2dc66dd16282 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:02.867801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2ab36-eb72-497e-a090-2dc66dd16282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:02.868939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1242/2480] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:02 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:20:02.889573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88193b-bd0b-49d7-afbb-10322810a8c3 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:02.891084 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3a88193b-bd0b-49d7-afbb-10322810a8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:02.892502 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9cb2ab36-eb72-497e-a090-2dc66dd16282 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:02.894375 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9cb2ab36-eb72-497e-a090-2dc66dd16282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:02.907307 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1239/2481] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:20:02.910477 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2ab36-eb72-497e-a090-2dc66dd16282 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:02.921119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2ab36-eb72-497e-a090-2dc66dd16282 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:02.927121 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3459f-2c61-490c-aee1-bc4678589a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:02.927797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3459f-2c61-490c-aee1-bc4678589a9c admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:02.928102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3459f-2c61-490c-aee1-bc4678589a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:02.928419 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b3459f-2c61-490c-aee1-bc4678589a9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:02.929178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3459f-2c61-490c-aee1-bc4678589a9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:20:02.929515 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3459f-2c61-490c-aee1-bc4678589a9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:02.931274 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b3459f-2c61-490c-aee1-bc4678589a9c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:02.950422 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1240/2482] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:20:02.951768 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3459f-2c61-490c-aee1-bc4678589a9c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:02.964852 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b3459f-2c61-490c-aee1-bc4678589a9c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:02.970910 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa4dea-45de-4229-b35c-2f125ccbb86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:02.972530 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa4dea-45de-4229-b35c-2f125ccbb86d admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:02.972988 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa4dea-45de-4229-b35c-2f125ccbb86d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:02.973420 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa4dea-45de-4229-b35c-2f125ccbb86d admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:02.974307 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa4dea-45de-4229-b35c-2f125ccbb86d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:02.976446 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa4dea-45de-4229-b35c-2f125ccbb86d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:03.022057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1241/2483] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:20:02 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:20:03.023765 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa4dea-45de-4229-b35c-2f125ccbb86d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:03.039079 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88193b-bd0b-49d7-afbb-10322810a8c3 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:03.073822 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:03.099553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa4dea-45de-4229-b35c-2f125ccbb86d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:03.104608 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c19f66-2baf-4bc9-8d20-743caa2a7219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:03.105229 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c19f66-2baf-4bc9-8d20-743caa2a7219 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:03.105516 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c19f66-2baf-4bc9-8d20-743caa2a7219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:03.105797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c19f66-2baf-4bc9-8d20-743caa2a7219 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:03.106448 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c19f66-2baf-4bc9-8d20-743caa2a7219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:03.108851 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c19f66-2baf-4bc9-8d20-743caa2a7219 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:03.114801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db97a9-68da-4ecd-81e3-ee8f01ba7f2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:03.115470 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db97a9-68da-4ecd-81e3-ee8f01ba7f2f service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:03.115752 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db97a9-68da-4ecd-81e3-ee8f01ba7f2f service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:03.116026 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db97a9-68da-4ecd-81e3-ee8f01ba7f2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:03.117800 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-64db97a9-68da-4ecd-81e3-ee8f01ba7f2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:03.135425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1242/2484] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:20:03 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:20:03.139617 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c19f66-2baf-4bc9-8d20-743caa2a7219 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:03.152334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c19f66-2baf-4bc9-8d20-743caa2a7219 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:03.158340 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff13822-b626-47aa-8d31-09162a1b1e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:03.159631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff13822-b626-47aa-8d31-09162a1b1e18 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:03.160558 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff13822-b626-47aa-8d31-09162a1b1e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:03.160838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff13822-b626-47aa-8d31-09162a1b1e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:03.161509 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff13822-b626-47aa-8d31-09162a1b1e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:03.163130 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff13822-b626-47aa-8d31-09162a1b1e18 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:03.176367 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1243/2485] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:20:03.180747 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff13822-b626-47aa-8d31-09162a1b1e18 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:03.192622 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db97a9-68da-4ecd-81e3-ee8f01ba7f2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:03.193199 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff13822-b626-47aa-8d31-09162a1b1e18 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:03.194250 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db97a9-68da-4ecd-81e3-ee8f01ba7f2f service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:03.198348 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e22387-7a89-4533-bd53-aa45ff2d84f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:03.199020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e22387-7a89-4533-bd53-aa45ff2d84f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:03.199625 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e22387-7a89-4533-bd53-aa45ff2d84f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:03.199907 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e22387-7a89-4533-bd53-aa45ff2d84f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:03.200561 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e22387-7a89-4533-bd53-aa45ff2d84f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:03.202237 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e22387-7a89-4533-bd53-aa45ff2d84f1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:03.205122 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1243/2486] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:20:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:20:03.214581 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1244/2487] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:20:03.218673 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db97a9-68da-4ecd-81e3-ee8f01ba7f2f service nova] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:03.229794 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db97a9-68da-4ecd-81e3-ee8f01ba7f2f service nova] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:03.236275 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7634a73-8277-47d5-bbf1-e028f7f9f43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:03.237003 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7634a73-8277-47d5-bbf1-e028f7f9f43b admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:03.237897 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7634a73-8277-47d5-bbf1-e028f7f9f43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:03.238270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7634a73-8277-47d5-bbf1-e028f7f9f43b admin admin] PATH_INFO: `/v3/projects/33ff321a41ef45c3a36861e6569cad06/users/c03bcd9b4025430997a296504c130ded/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:03.255398 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7634a73-8277-47d5-bbf1-e028f7f9f43b admin admin] RBAC: Authorizing `identity:create_grant(project_id=33ff321a41ef45c3a36861e6569cad06, user_id=c03bcd9b4025430997a296504c130ded, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:03.258689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7634a73-8277-47d5-bbf1-e028f7f9f43b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:03.284794 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1244/2488] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:20:03 2022] PUT /identity/v3/projects/33ff321a41ef45c3a36861e6569cad06/users/c03bcd9b4025430997a296504c130ded/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:20:03.296332 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c756e9-42c1-4d45-91b3-7ecc4fedabbd None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:03.296627 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c756e9-42c1-4d45-91b3-7ecc4fedabbd None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:03.296915 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c756e9-42c1-4d45-91b3-7ecc4fedabbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:03.338738 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-29c756e9-42c1-4d45-91b3-7ecc4fedabbd None None] MFA Rules not processed for user `c03bcd9b4025430997a296504c130ded`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:03.340132 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-29c756e9-42c1-4d45-91b3-7ecc4fedabbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:03.423235 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:03.448877 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1245/2489] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:20:03.458048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7634a73-8277-47d5-bbf1-e028f7f9f43b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:03.461052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-b7634a73-8277-47d5-bbf1-e028f7f9f43b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:03.526953 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7634a73-8277-47d5-bbf1-e028f7f9f43b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:03.530686 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4076029-1059-4355-93ba-9305e90eadd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:03.531334 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4076029-1059-4355-93ba-9305e90eadd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:03.531608 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4076029-1059-4355-93ba-9305e90eadd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:03.531881 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4076029-1059-4355-93ba-9305e90eadd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:03.541564 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4076029-1059-4355-93ba-9305e90eadd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:03.543069 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4076029-1059-4355-93ba-9305e90eadd6 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:03.555032 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1245/2490] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:20:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:20:03.591643 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c756e9-42c1-4d45-91b3-7ecc4fedabbd None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:03.669916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c756e9-42c1-4d45-91b3-7ecc4fedabbd None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:03.677120 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834d533-6f37-4c0f-b9a1-3db602960903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:03.677865 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834d533-6f37-4c0f-b9a1-3db602960903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:03.678212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834d533-6f37-4c0f-b9a1-3db602960903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:03.678555 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8834d533-6f37-4c0f-b9a1-3db602960903 admin admin] PATH_INFO: `/v3/users/c03bcd9b4025430997a296504c130ded` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:03.680286 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834d533-6f37-4c0f-b9a1-3db602960903 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c03bcd9b4025430997a296504c130ded)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:03.682576 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8834d533-6f37-4c0f-b9a1-3db602960903 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:03.744008 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-8834d533-6f37-4c0f-b9a1-3db602960903 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c03bcd9b4025430997a296504c130ded'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:03.752241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-8834d533-6f37-4c0f-b9a1-3db602960903 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c03bcd9b4025430997a296504c130ded'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:03.753534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-8834d533-6f37-4c0f-b9a1-3db602960903 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c03bcd9b4025430997a296504c130ded'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:03.762647 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-8834d533-6f37-4c0f-b9a1-3db602960903 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c03bcd9b4025430997a296504c130ded'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:03.777804 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-8834d533-6f37-4c0f-b9a1-3db602960903 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c03bcd9b4025430997a296504c130ded'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:03.786690 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1246/2491] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:20:03 2022] DELETE /identity/v3/users/c03bcd9b4025430997a296504c130ded => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:20:03.792575 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a021f-0e81-402a-b15d-5c498347c167 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:03.793065 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a021f-0e81-402a-b15d-5c498347c167 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:03.793375 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a021f-0e81-402a-b15d-5c498347c167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:03.815820 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-a51a021f-0e81-402a-b15d-5c498347c167 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:03.817146 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a51a021f-0e81-402a-b15d-5c498347c167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:03.928677 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:03.965438 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834d533-6f37-4c0f-b9a1-3db602960903 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:03.967826 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8834d533-6f37-4c0f-b9a1-3db602960903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:03.970921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1246/2492] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:20:03.981406 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a021f-0e81-402a-b15d-5c498347c167 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:03.985135 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8834d533-6f37-4c0f-b9a1-3db602960903 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:03.987986 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-a51a021f-0e81-402a-b15d-5c498347c167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:03.990937 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e2f5d-53a1-42a8-b520-ee745fdd9c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:03.991675 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e2f5d-53a1-42a8-b520-ee745fdd9c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:03.992030 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e2f5d-53a1-42a8-b520-ee745fdd9c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:03.992376 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e2f5d-53a1-42a8-b520-ee745fdd9c0e admin admin] PATH_INFO: `/v3/users/b114171a48d04ba1b21d8a1e19ee8463` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:03.994428 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e2f5d-53a1-42a8-b520-ee745fdd9c0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b114171a48d04ba1b21d8a1e19ee8463)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:03.996403 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e2f5d-53a1-42a8-b520-ee745fdd9c0e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:04.092004 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-df1e2f5d-53a1-42a8-b520-ee745fdd9c0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b114171a48d04ba1b21d8a1e19ee8463'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:04.092950 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a021f-0e81-402a-b15d-5c498347c167 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:04.097685 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafc992-2521-4226-8430-3c16622175ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:04.098907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafc992-2521-4226-8430-3c16622175ee service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:04.099254 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafc992-2521-4226-8430-3c16622175ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:04.099534 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafc992-2521-4226-8430-3c16622175ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:04.100969 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-df1e2f5d-53a1-42a8-b520-ee745fdd9c0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b114171a48d04ba1b21d8a1e19ee8463'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:04.102727 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-df1e2f5d-53a1-42a8-b520-ee745fdd9c0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b114171a48d04ba1b21d8a1e19ee8463'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:04.107307 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8cafc992-2521-4226-8430-3c16622175ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:04.112775 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-df1e2f5d-53a1-42a8-b520-ee745fdd9c0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b114171a48d04ba1b21d8a1e19ee8463'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:04.120460 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:04.123401 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafc992-2521-4226-8430-3c16622175ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:04.124891 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafc992-2521-4226-8430-3c16622175ee service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:04.131797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-df1e2f5d-53a1-42a8-b520-ee745fdd9c0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b114171a48d04ba1b21d8a1e19ee8463'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:04.138393 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1247/2493] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:20:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:20:04.140038 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1247/2494] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:20:03 2022] DELETE /identity/v3/users/b114171a48d04ba1b21d8a1e19ee8463 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:20:04.144653 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b49ae-97c8-42d4-9523-cbeb55b3019b None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:04.144935 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b49ae-97c8-42d4-9523-cbeb55b3019b None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:04.145205 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b49ae-97c8-42d4-9523-cbeb55b3019b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:04.165701 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-089b49ae-97c8-42d4-9523-cbeb55b3019b None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:04.166921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-089b49ae-97c8-42d4-9523-cbeb55b3019b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:04.239762 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1248/2495] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:20:04.251984 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e2f5d-53a1-42a8-b520-ee745fdd9c0e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:04.254714 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-df1e2f5d-53a1-42a8-b520-ee745fdd9c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:04.349222 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e2f5d-53a1-42a8-b520-ee745fdd9c0e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:04.355634 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eed62f-5356-44e2-aa4d-4f981d7832a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:04.357077 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eed62f-5356-44e2-aa4d-4f981d7832a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:04.357686 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eed62f-5356-44e2-aa4d-4f981d7832a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:04.358186 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eed62f-5356-44e2-aa4d-4f981d7832a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:04.368866 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eed62f-5356-44e2-aa4d-4f981d7832a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:04.370406 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eed62f-5356-44e2-aa4d-4f981d7832a0 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:04.384043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1248/2496] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:20:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:20:04.616376 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b49ae-97c8-42d4-9523-cbeb55b3019b None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:04.617827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-089b49ae-97c8-42d4-9523-cbeb55b3019b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:04.633342 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b49ae-97c8-42d4-9523-cbeb55b3019b None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:04.638436 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cd684-1637-4298-860f-58819838788e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:04.639123 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cd684-1637-4298-860f-58819838788e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:04.639407 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cd684-1637-4298-860f-58819838788e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:04.639682 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cd684-1637-4298-860f-58819838788e admin admin] PATH_INFO: `/v3/projects/33ff321a41ef45c3a36861e6569cad06` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:04.640926 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cd684-1637-4298-860f-58819838788e admin admin] RBAC: Authorizing `identity:delete_project(project_id=33ff321a41ef45c3a36861e6569cad06)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:04.642857 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cd684-1637-4298-860f-58819838788e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:04.717170 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-703cd684-1637-4298-860f-58819838788e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33ff321a41ef45c3a36861e6569cad06', 'request_id': 'req-703cd684-1637-4298-860f-58819838788e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:04.726105 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-703cd684-1637-4298-860f-58819838788e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33ff321a41ef45c3a36861e6569cad06', 'request_id': 'req-703cd684-1637-4298-860f-58819838788e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:04.735071 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-703cd684-1637-4298-860f-58819838788e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33ff321a41ef45c3a36861e6569cad06', 'request_id': 'req-703cd684-1637-4298-860f-58819838788e'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:04.765979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-703cd684-1637-4298-860f-58819838788e admin admin] The token cache is being invalidate because project 33ff321a41ef45c3a36861e6569cad06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:20:04.765979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-703cd684-1637-4298-860f-58819838788e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:04.767394 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-703cd684-1637-4298-860f-58819838788e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:04.772038 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1249/2497] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:20:04 2022] DELETE /identity/v3/projects/33ff321a41ef45c3a36861e6569cad06 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:20:04.778599 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a222cd-8510-4345-a9a1-afc7d8b66866 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:04.779149 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a222cd-8510-4345-a9a1-afc7d8b66866 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:04.779242 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a222cd-8510-4345-a9a1-afc7d8b66866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:04.832398 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cd684-1637-4298-860f-58819838788e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:04.833881 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a6a222cd-8510-4345-a9a1-afc7d8b66866 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:04.834094 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-703cd684-1637-4298-860f-58819838788e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:04.835493 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a6a222cd-8510-4345-a9a1-afc7d8b66866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:04.862894 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:04.933676 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cd684-1637-4298-860f-58819838788e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:04.936875 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1249/2498] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:20:04.945169 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3837020-7134-4293-875a-e78cd9e638ed None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:04.945534 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3837020-7134-4293-875a-e78cd9e638ed None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:04.945814 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230c781-456f-4e6d-b19f-514a4c6b787f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:04.945962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3837020-7134-4293-875a-e78cd9e638ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:04.946912 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230c781-456f-4e6d-b19f-514a4c6b787f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:04.947672 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230c781-456f-4e6d-b19f-514a4c6b787f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:04.948265 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230c781-456f-4e6d-b19f-514a4c6b787f admin admin] PATH_INFO: `/v3/projects/91c56fdfcf1948f9aab685ce5b1b9cbe` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:04.950091 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230c781-456f-4e6d-b19f-514a4c6b787f admin admin] RBAC: Authorizing `identity:delete_project(project_id=91c56fdfcf1948f9aab685ce5b1b9cbe)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:04.953483 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230c781-456f-4e6d-b19f-514a4c6b787f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:04.973101 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e3837020-7134-4293-875a-e78cd9e638ed None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:04.975065 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e3837020-7134-4293-875a-e78cd9e638ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:04.989283 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1250/2499] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:20:04.990312 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3837020-7134-4293-875a-e78cd9e638ed None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:05.021976 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3837020-7134-4293-875a-e78cd9e638ed None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:05.036198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8306d-2a3b-44f0-bcf5-01462b45fe75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:05.036802 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8306d-2a3b-44f0-bcf5-01462b45fe75 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:05.037115 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8306d-2a3b-44f0-bcf5-01462b45fe75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:05.037407 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8306d-2a3b-44f0-bcf5-01462b45fe75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:05.038062 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8306d-2a3b-44f0-bcf5-01462b45fe75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:20:05.038678 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8306d-2a3b-44f0-bcf5-01462b45fe75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:05.040309 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8306d-2a3b-44f0-bcf5-01462b45fe75 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:05.075081 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1251/2500] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:20:05.076573 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8306d-2a3b-44f0-bcf5-01462b45fe75 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:05.077553 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2230c781-456f-4e6d-b19f-514a4c6b787f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91c56fdfcf1948f9aab685ce5b1b9cbe', 'request_id': 'req-2230c781-456f-4e6d-b19f-514a4c6b787f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:05.093068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8306d-2a3b-44f0-bcf5-01462b45fe75 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:05.093732 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2230c781-456f-4e6d-b19f-514a4c6b787f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91c56fdfcf1948f9aab685ce5b1b9cbe', 'request_id': 'req-2230c781-456f-4e6d-b19f-514a4c6b787f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:05.102253 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2230c781-456f-4e6d-b19f-514a4c6b787f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91c56fdfcf1948f9aab685ce5b1b9cbe', 'request_id': 'req-2230c781-456f-4e6d-b19f-514a4c6b787f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:05.122532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2230c781-456f-4e6d-b19f-514a4c6b787f admin admin] The token cache is being invalidate because project 91c56fdfcf1948f9aab685ce5b1b9cbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:20:05.123194 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2230c781-456f-4e6d-b19f-514a4c6b787f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:05.124121 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-2230c781-456f-4e6d-b19f-514a4c6b787f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:05.126289 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1250/2501] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:20:04 2022] DELETE /identity/v3/projects/91c56fdfcf1948f9aab685ce5b1b9cbe => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:20:05.131382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230c781-456f-4e6d-b19f-514a4c6b787f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:05.133028 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-2230c781-456f-4e6d-b19f-514a4c6b787f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:05.165020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc9d5a-50af-41ce-97bc-0f4f68005309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:05.166106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc9d5a-50af-41ce-97bc-0f4f68005309 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:05.166106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc9d5a-50af-41ce-97bc-0f4f68005309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:05.166328 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc9d5a-50af-41ce-97bc-0f4f68005309 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:05.166801 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc9d5a-50af-41ce-97bc-0f4f68005309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:05.168959 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc9d5a-50af-41ce-97bc-0f4f68005309 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:05.189732 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:05.205404 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1252/2502] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:20:05 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:20:05.209488 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc9d5a-50af-41ce-97bc-0f4f68005309 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:05.211584 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1edc9d5a-50af-41ce-97bc-0f4f68005309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:05.218343 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230c781-456f-4e6d-b19f-514a4c6b787f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:05.285485 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc9d5a-50af-41ce-97bc-0f4f68005309 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:05.285897 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:05.292984 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c5d60-cd2b-4f3f-9545-898ee21076f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:05.293649 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c5d60-cd2b-4f3f-9545-898ee21076f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:05.293968 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c5d60-cd2b-4f3f-9545-898ee21076f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:05.294212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c5d60-cd2b-4f3f-9545-898ee21076f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:05.294926 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c5d60-cd2b-4f3f-9545-898ee21076f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:05.296776 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c5d60-cd2b-4f3f-9545-898ee21076f8 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:05.301800 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cddba41-6bba-49bf-bdc4-30beca63a53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:05.302463 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddba41-6bba-49bf-bdc4-30beca63a53e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:05.302786 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddba41-6bba-49bf-bdc4-30beca63a53e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:05.303076 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddba41-6bba-49bf-bdc4-30beca63a53e admin admin] PATH_INFO: `/v3/users/60ef1c2f23144fa1a24ed824420ea3d3` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:05.304310 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cddba41-6bba-49bf-bdc4-30beca63a53e admin admin] RBAC: Authorizing `identity:delete_user(user_id=60ef1c2f23144fa1a24ed824420ea3d3)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:05.306358 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cddba41-6bba-49bf-bdc4-30beca63a53e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:05.327262 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1253/2503] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:20:05 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:20:05.340575 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c5d60-cd2b-4f3f-9545-898ee21076f8 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:05.357532 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c5d60-cd2b-4f3f-9545-898ee21076f8 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:05.364697 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c02da-ab32-4a94-8333-800b15f2c937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:05.365394 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c02da-ab32-4a94-8333-800b15f2c937 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:05.365746 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c02da-ab32-4a94-8333-800b15f2c937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:05.366080 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c02da-ab32-4a94-8333-800b15f2c937 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:05.366893 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c02da-ab32-4a94-8333-800b15f2c937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:05.368738 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c02da-ab32-4a94-8333-800b15f2c937 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:05.381975 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1254/2504] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:20:05.385827 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c02da-ab32-4a94-8333-800b15f2c937 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:05.391737 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5cddba41-6bba-49bf-bdc4-30beca63a53e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60ef1c2f23144fa1a24ed824420ea3d3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:05.396629 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c02da-ab32-4a94-8333-800b15f2c937 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:05.399822 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5cddba41-6bba-49bf-bdc4-30beca63a53e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60ef1c2f23144fa1a24ed824420ea3d3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:05.400576 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5cddba41-6bba-49bf-bdc4-30beca63a53e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60ef1c2f23144fa1a24ed824420ea3d3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:05.402294 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19562934-8139-45da-ac95-fa88d2669c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:05.402944 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19562934-8139-45da-ac95-fa88d2669c49 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:05.403234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19562934-8139-45da-ac95-fa88d2669c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:05.403515 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19562934-8139-45da-ac95-fa88d2669c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:05.404235 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19562934-8139-45da-ac95-fa88d2669c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:05.406063 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19562934-8139-45da-ac95-fa88d2669c49 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:05.412224 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5cddba41-6bba-49bf-bdc4-30beca63a53e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60ef1c2f23144fa1a24ed824420ea3d3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:05.424550 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1255/2505] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:20:05.426058 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19562934-8139-45da-ac95-fa88d2669c49 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:05.428022 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-19562934-8139-45da-ac95-fa88d2669c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:05.429154 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-5cddba41-6bba-49bf-bdc4-30beca63a53e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60ef1c2f23144fa1a24ed824420ea3d3'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:05.437530 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1251/2506] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:20:05 2022] DELETE /identity/v3/users/60ef1c2f23144fa1a24ed824420ea3d3 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:20:05.458040 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19562934-8139-45da-ac95-fa88d2669c49 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:05.583292 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a221254-2c46-4f63-9ada-0c3211c9fd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:05.584039 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a221254-2c46-4f63-9ada-0c3211c9fd97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:05.584328 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a221254-2c46-4f63-9ada-0c3211c9fd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:05.584606 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a221254-2c46-4f63-9ada-0c3211c9fd97 admin admin] PATH_INFO: `/v3/projects/b681923b7d7b4cf682ae41fdce65a823/users/1ba9dfbbcb5c424da17e7b198aef35d9/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:05.606739 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a221254-2c46-4f63-9ada-0c3211c9fd97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b681923b7d7b4cf682ae41fdce65a823, user_id=1ba9dfbbcb5c424da17e7b198aef35d9, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:05.609761 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a221254-2c46-4f63-9ada-0c3211c9fd97 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:05.626338 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1256/2507] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:20:05 2022] PUT /identity/v3/projects/b681923b7d7b4cf682ae41fdce65a823/users/1ba9dfbbcb5c424da17e7b198aef35d9/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:20:05.637717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7498b-0521-4a55-8a51-0c057c6462d5 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:05.638080 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7498b-0521-4a55-8a51-0c057c6462d5 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:05.638557 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7498b-0521-4a55-8a51-0c057c6462d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:05.678000 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f2d7498b-0521-4a55-8a51-0c057c6462d5 None None] MFA Rules not processed for user `1ba9dfbbcb5c424da17e7b198aef35d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:05.680176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f2d7498b-0521-4a55-8a51-0c057c6462d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:05.764875 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1252/2508] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:20:05.776705 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a221254-2c46-4f63-9ada-0c3211c9fd97 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:05.778901 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5a221254-2c46-4f63-9ada-0c3211c9fd97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:05.791269 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7498b-0521-4a55-8a51-0c057c6462d5 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:05.792833 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f2d7498b-0521-4a55-8a51-0c057c6462d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:05.859427 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:05.868876 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a221254-2c46-4f63-9ada-0c3211c9fd97 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:05.873325 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289b1ab-72ac-4313-8a96-ed1053c82311 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:05.874514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289b1ab-72ac-4313-8a96-ed1053c82311 service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:05.874514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289b1ab-72ac-4313-8a96-ed1053c82311 service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:05.875119 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289b1ab-72ac-4313-8a96-ed1053c82311 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:05.886094 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2289b1ab-72ac-4313-8a96-ed1053c82311 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:05.887766 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2289b1ab-72ac-4313-8a96-ed1053c82311 service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:05.889773 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7498b-0521-4a55-8a51-0c057c6462d5 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:05.897996 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db6a1f-e073-4ba5-a519-b9416aa30015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:05.898710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1257/2509] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:20:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:20:05.899153 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db6a1f-e073-4ba5-a519-b9416aa30015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:05.899690 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db6a1f-e073-4ba5-a519-b9416aa30015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:05.900114 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db6a1f-e073-4ba5-a519-b9416aa30015 admin admin] PATH_INFO: `/v3/projects/c41cf62e5be946c5ad61a51dd0bf90f1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:05.901419 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db6a1f-e073-4ba5-a519-b9416aa30015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c41cf62e5be946c5ad61a51dd0bf90f1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:05.904017 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db6a1f-e073-4ba5-a519-b9416aa30015 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:05.908190 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:05.985244 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c8db6a1f-e073-4ba5-a519-b9416aa30015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c41cf62e5be946c5ad61a51dd0bf90f1', 'request_id': 'req-c8db6a1f-e073-4ba5-a519-b9416aa30015'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:05.992524 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c8db6a1f-e073-4ba5-a519-b9416aa30015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c41cf62e5be946c5ad61a51dd0bf90f1', 'request_id': 'req-c8db6a1f-e073-4ba5-a519-b9416aa30015'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:06.001238 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c8db6a1f-e073-4ba5-a519-b9416aa30015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c41cf62e5be946c5ad61a51dd0bf90f1', 'request_id': 'req-c8db6a1f-e073-4ba5-a519-b9416aa30015'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:06.021207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c8db6a1f-e073-4ba5-a519-b9416aa30015 admin admin] The token cache is being invalidate because project c41cf62e5be946c5ad61a51dd0bf90f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:20:06.021532 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c8db6a1f-e073-4ba5-a519-b9416aa30015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:06.022361 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c8db6a1f-e073-4ba5-a519-b9416aa30015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:06.024366 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1253/2510] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:20:05 2022] DELETE /identity/v3/projects/c41cf62e5be946c5ad61a51dd0bf90f1 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:20:06.028497 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289b1ab-72ac-4313-8a96-ed1053c82311 service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:06.044651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289b1ab-72ac-4313-8a96-ed1053c82311 service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:06.136509 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f343d7-116e-4931-b25d-e26fea6ee73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:06.137233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f343d7-116e-4931-b25d-e26fea6ee73b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:06.137858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f343d7-116e-4931-b25d-e26fea6ee73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:06.138203 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f343d7-116e-4931-b25d-e26fea6ee73b admin admin] PATH_INFO: `/v3/users/dae6a7ed1d2b4423bad1237a8976be84` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:06.139619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f343d7-116e-4931-b25d-e26fea6ee73b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dae6a7ed1d2b4423bad1237a8976be84)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:06.142150 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f343d7-116e-4931-b25d-e26fea6ee73b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:06.204857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-62f343d7-116e-4931-b25d-e26fea6ee73b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dae6a7ed1d2b4423bad1237a8976be84'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:06.213015 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-62f343d7-116e-4931-b25d-e26fea6ee73b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dae6a7ed1d2b4423bad1237a8976be84'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:06.213993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-62f343d7-116e-4931-b25d-e26fea6ee73b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dae6a7ed1d2b4423bad1237a8976be84'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:06.226502 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-62f343d7-116e-4931-b25d-e26fea6ee73b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dae6a7ed1d2b4423bad1237a8976be84'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:06.241980 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-62f343d7-116e-4931-b25d-e26fea6ee73b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dae6a7ed1d2b4423bad1237a8976be84'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:06.256309 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1258/2511] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:20:06 2022] DELETE /identity/v3/users/dae6a7ed1d2b4423bad1237a8976be84 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:20:06.605618 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db6a1f-e073-4ba5-a519-b9416aa30015 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:06.607529 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c8db6a1f-e073-4ba5-a519-b9416aa30015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:06.622400 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db6a1f-e073-4ba5-a519-b9416aa30015 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:06.689431 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb53b1-3af1-4685-a8e2-76c11fec9c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:06.690002 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb53b1-3af1-4685-a8e2-76c11fec9c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:06.690351 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb53b1-3af1-4685-a8e2-76c11fec9c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:06.690665 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb53b1-3af1-4685-a8e2-76c11fec9c3c admin admin] PATH_INFO: `/v3/projects/0ad6546fe22540dcb26c81395481ec65` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:06.691893 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb53b1-3af1-4685-a8e2-76c11fec9c3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ad6546fe22540dcb26c81395481ec65)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:06.693860 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb53b1-3af1-4685-a8e2-76c11fec9c3c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:06.748207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-79fb53b1-3af1-4685-a8e2-76c11fec9c3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ad6546fe22540dcb26c81395481ec65', 'request_id': 'req-79fb53b1-3af1-4685-a8e2-76c11fec9c3c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:06.753738 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-79fb53b1-3af1-4685-a8e2-76c11fec9c3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ad6546fe22540dcb26c81395481ec65', 'request_id': 'req-79fb53b1-3af1-4685-a8e2-76c11fec9c3c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:06.761269 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-79fb53b1-3af1-4685-a8e2-76c11fec9c3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ad6546fe22540dcb26c81395481ec65', 'request_id': 'req-79fb53b1-3af1-4685-a8e2-76c11fec9c3c'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:06.775259 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-79fb53b1-3af1-4685-a8e2-76c11fec9c3c admin admin] The token cache is being invalidate because project 0ad6546fe22540dcb26c81395481ec65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:20:06.775553 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-79fb53b1-3af1-4685-a8e2-76c11fec9c3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:06.776267 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-79fb53b1-3af1-4685-a8e2-76c11fec9c3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:06.777947 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1254/2512] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:20:06 2022] DELETE /identity/v3/projects/0ad6546fe22540dcb26c81395481ec65 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:20:06.785433 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7bf96-df79-4323-b892-ddb270f25ec4 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:06.785962 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7bf96-df79-4323-b892-ddb270f25ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:06.786072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7bf96-df79-4323-b892-ddb270f25ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:06.808239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-57a7bf96-df79-4323-b892-ddb270f25ec4 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:06.809473 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-57a7bf96-df79-4323-b892-ddb270f25ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:06.875853 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1259/2513] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:20:06.884546 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd491c8-a6a9-4b56-8583-7474ed24a30c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:06.884836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd491c8-a6a9-4b56-8583-7474ed24a30c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:06.885116 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd491c8-a6a9-4b56-8583-7474ed24a30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:06.904783 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-bfd491c8-a6a9-4b56-8583-7474ed24a30c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:06.906042 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-bfd491c8-a6a9-4b56-8583-7474ed24a30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:06.914874 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1255/2514] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:20:06.919136 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7bf96-df79-4323-b892-ddb270f25ec4 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:06.929452 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7bf96-df79-4323-b892-ddb270f25ec4 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:06.933960 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6cc2b6-bd73-4087-a146-c3165ac5029d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:06.934579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6cc2b6-bd73-4087-a146-c3165ac5029d admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:06.934855 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6cc2b6-bd73-4087-a146-c3165ac5029d admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:06.935160 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6cc2b6-bd73-4087-a146-c3165ac5029d admin admin] PATH_INFO: `/v3/domains` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:06.935803 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6cc2b6-bd73-4087-a146-c3165ac5029d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:20:06.936102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6cc2b6-bd73-4087-a146-c3165ac5029d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:06.937720 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6cc2b6-bd73-4087-a146-c3165ac5029d admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:06.940406 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:06.951466 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1260/2515] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:20:06.955441 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd491c8-a6a9-4b56-8583-7474ed24a30c None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:06.964923 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd491c8-a6a9-4b56-8583-7474ed24a30c None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:06.969634 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd6221-6ed8-402f-9bf8-10db85e0f758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:06.970246 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd6221-6ed8-402f-9bf8-10db85e0f758 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:06.970530 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd6221-6ed8-402f-9bf8-10db85e0f758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:06.970807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd6221-6ed8-402f-9bf8-10db85e0f758 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:06.971996 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd6221-6ed8-402f-9bf8-10db85e0f758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:06.973916 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd6221-6ed8-402f-9bf8-10db85e0f758 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:06.989442 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:07.004907 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1256/2516] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:20:06 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:20:07.009102 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6cc2b6-bd73-4087-a146-c3165ac5029d admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:07.020013 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6cc2b6-bd73-4087-a146-c3165ac5029d admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:07.086003 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c8153-77d3-4b0d-8f1e-d34fb6a881ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:07.086621 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c8153-77d3-4b0d-8f1e-d34fb6a881ad admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:07.086927 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c8153-77d3-4b0d-8f1e-d34fb6a881ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:07.087202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c8153-77d3-4b0d-8f1e-d34fb6a881ad admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:07.087847 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c8153-77d3-4b0d-8f1e-d34fb6a881ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:07.089755 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c8153-77d3-4b0d-8f1e-d34fb6a881ad admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:07.116890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1261/2517] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:20:07 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:20:07.121337 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd6221-6ed8-402f-9bf8-10db85e0f758 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:07.132224 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd6221-6ed8-402f-9bf8-10db85e0f758 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:07.140080 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380f651-2296-45b8-853a-923e567dcc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:07.140736 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380f651-2296-45b8-853a-923e567dcc08 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:07.140999 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380f651-2296-45b8-853a-923e567dcc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:07.141282 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380f651-2296-45b8-853a-923e567dcc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:07.141945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380f651-2296-45b8-853a-923e567dcc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:07.143650 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380f651-2296-45b8-853a-923e567dcc08 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:07.154382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1257/2518] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:20:07.158429 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c8153-77d3-4b0d-8f1e-d34fb6a881ad admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:07.170831 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c8153-77d3-4b0d-8f1e-d34fb6a881ad admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:07.177020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60068f2d-31d0-46c3-8e2c-26c10ead5c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:07.177706 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60068f2d-31d0-46c3-8e2c-26c10ead5c5b admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:07.177966 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60068f2d-31d0-46c3-8e2c-26c10ead5c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:07.178301 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60068f2d-31d0-46c3-8e2c-26c10ead5c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:07.179128 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60068f2d-31d0-46c3-8e2c-26c10ead5c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:07.180858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60068f2d-31d0-46c3-8e2c-26c10ead5c5b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:07.196504 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1262/2519] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:20:07.197982 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380f651-2296-45b8-853a-923e567dcc08 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:07.209072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380f651-2296-45b8-853a-923e567dcc08 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:07.213976 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e6527-433b-471b-a6e0-d3e24deb6252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:07.214942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e6527-433b-471b-a6e0-d3e24deb6252 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:07.215231 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e6527-433b-471b-a6e0-d3e24deb6252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:07.215541 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e6527-433b-471b-a6e0-d3e24deb6252 admin admin] PATH_INFO: `/v3/projects/de3faeed7ce64fa89d85fadeafc9bdf5/users/45c529adbb514287999752cb54bba689/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:07.235801 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e6527-433b-471b-a6e0-d3e24deb6252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de3faeed7ce64fa89d85fadeafc9bdf5, user_id=45c529adbb514287999752cb54bba689, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:07.238714 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e6527-433b-471b-a6e0-d3e24deb6252 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:07.261848 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1258/2520] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:20:07 2022] PUT /identity/v3/projects/de3faeed7ce64fa89d85fadeafc9bdf5/users/45c529adbb514287999752cb54bba689/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:20:07.279913 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc75f5-aaef-4782-adeb-95ef0d3fab2e None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:07.279913 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc75f5-aaef-4782-adeb-95ef0d3fab2e None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:07.279913 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc75f5-aaef-4782-adeb-95ef0d3fab2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:07.311036 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9fbc75f5-aaef-4782-adeb-95ef0d3fab2e None None] MFA Rules not processed for user `45c529adbb514287999752cb54bba689`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:07.312216 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9fbc75f5-aaef-4782-adeb-95ef0d3fab2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:07.389779 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1263/2521] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:20:07.394915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e6527-433b-471b-a6e0-d3e24deb6252 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:07.400607 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f1c8f-5f36-4e08-a009-222a970f88d7 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:07.401072 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f1c8f-5f36-4e08-a009-222a970f88d7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:07.401414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f1c8f-5f36-4e08-a009-222a970f88d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:07.431688 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-950f1c8f-5f36-4e08-a009-222a970f88d7 None None] MFA Rules not processed for user `1ba9dfbbcb5c424da17e7b198aef35d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:07.433320 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-950f1c8f-5f36-4e08-a009-222a970f88d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:07.493569 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e6527-433b-471b-a6e0-d3e24deb6252 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:07.499104 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d9c1b-8796-4359-9057-5b4d3cee0a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:07.499722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d9c1b-8796-4359-9057-5b4d3cee0a01 admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:07.500005 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d9c1b-8796-4359-9057-5b4d3cee0a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:07.500280 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d9c1b-8796-4359-9057-5b4d3cee0a01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:07.501137 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d9c1b-8796-4359-9057-5b4d3cee0a01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:07.503272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d9c1b-8796-4359-9057-5b4d3cee0a01 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:07.512694 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1264/2522] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:20:07.551598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1259/2523] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:20:07 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:20:07.552573 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f1c8f-5f36-4e08-a009-222a970f88d7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:07.564211 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f1c8f-5f36-4e08-a009-222a970f88d7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:07.573666 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:07.656814 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d127a09-1bdd-4185-a54b-eb2eaa80be67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:07.657441 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d127a09-1bdd-4185-a54b-eb2eaa80be67 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:07.657778 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d127a09-1bdd-4185-a54b-eb2eaa80be67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:07.658058 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d127a09-1bdd-4185-a54b-eb2eaa80be67 admin admin] PATH_INFO: `/v3/users` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:07.658759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d127a09-1bdd-4185-a54b-eb2eaa80be67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:07.661329 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d127a09-1bdd-4185-a54b-eb2eaa80be67 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:07.688027 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1265/2524] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:20:07 2022] POST /identity/v3/users => generated 299 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:20:07.691844 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d9c1b-8796-4359-9057-5b4d3cee0a01 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:07.704382 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d9c1b-8796-4359-9057-5b4d3cee0a01 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:07.711120 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50c9fc-cb5d-4b93-9c4c-bce99f36a96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:07.711939 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50c9fc-cb5d-4b93-9c4c-bce99f36a96a admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:07.712378 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50c9fc-cb5d-4b93-9c4c-bce99f36a96a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:07.712813 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50c9fc-cb5d-4b93-9c4c-bce99f36a96a admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:07.713665 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50c9fc-cb5d-4b93-9c4c-bce99f36a96a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:07.716236 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50c9fc-cb5d-4b93-9c4c-bce99f36a96a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:07.731262 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1260/2525] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:20:07.736044 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d127a09-1bdd-4185-a54b-eb2eaa80be67 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:07.749414 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d127a09-1bdd-4185-a54b-eb2eaa80be67 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:07.755486 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53487f0-1613-45cd-b700-f014ea58ee3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:07.756280 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53487f0-1613-45cd-b700-f014ea58ee3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:07.756670 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53487f0-1613-45cd-b700-f014ea58ee3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:07.757061 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53487f0-1613-45cd-b700-f014ea58ee3b admin admin] PATH_INFO: `/v3/projects/ae618ca249b045e2af0981b22d243a38/users/82a60b540142469e948e381ce8af58f5/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:07.777684 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53487f0-1613-45cd-b700-f014ea58ee3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae618ca249b045e2af0981b22d243a38, user_id=82a60b540142469e948e381ce8af58f5, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:07.780588 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53487f0-1613-45cd-b700-f014ea58ee3b admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:07.799140 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1266/2526] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:20:07 2022] PUT /identity/v3/projects/ae618ca249b045e2af0981b22d243a38/users/82a60b540142469e948e381ce8af58f5/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:20:07.803442 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50c9fc-cb5d-4b93-9c4c-bce99f36a96a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:07.813972 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50c9fc-cb5d-4b93-9c4c-bce99f36a96a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:07.880943 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8673f0-2eab-4b72-b7e7-a6c91fd829e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:07.881562 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8673f0-2eab-4b72-b7e7-a6c91fd829e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:07.881842 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8673f0-2eab-4b72-b7e7-a6c91fd829e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:07.882119 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8673f0-2eab-4b72-b7e7-a6c91fd829e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:07.882771 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8673f0-2eab-4b72-b7e7-a6c91fd829e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:07.884463 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8673f0-2eab-4b72-b7e7-a6c91fd829e8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:07.895216 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1261/2527] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:20:07.899514 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53487f0-1613-45cd-b700-f014ea58ee3b admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:07.913049 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53487f0-1613-45cd-b700-f014ea58ee3b admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:07.922759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bae9a-a97e-4bee-9487-d586b98c4c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:07.923500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bae9a-a97e-4bee-9487-d586b98c4c91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:07.923788 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bae9a-a97e-4bee-9487-d586b98c4c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:07.924066 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bae9a-a97e-4bee-9487-d586b98c4c91 admin admin] PATH_INFO: `/v3/projects/ae618ca249b045e2af0981b22d243a38/users/82a60b540142469e948e381ce8af58f5/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:07.926107 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85bae9a-a97e-4bee-9487-d586b98c4c91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae618ca249b045e2af0981b22d243a38, user_id=82a60b540142469e948e381ce8af58f5, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:07.928831 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85bae9a-a97e-4bee-9487-d586b98c4c91 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:07.939689 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1267/2528] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:20:07 2022] PUT /identity/v3/projects/ae618ca249b045e2af0981b22d243a38/users/82a60b540142469e948e381ce8af58f5/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:20:07.950729 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca6591-bb65-4d63-82e5-fd4dc074f3ca None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:07.951237 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca6591-bb65-4d63-82e5-fd4dc074f3ca None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:07.951326 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca6591-bb65-4d63-82e5-fd4dc074f3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:07.982325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-86ca6591-bb65-4d63-82e5-fd4dc074f3ca None None] MFA Rules not processed for user `82a60b540142469e948e381ce8af58f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:07.983632 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-86ca6591-bb65-4d63-82e5-fd4dc074f3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:08.000920 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:08.070900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1262/2529] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:20:08.083188 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bae9a-a97e-4bee-9487-d586b98c4c91 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:08.085063 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c85bae9a-a97e-4bee-9487-d586b98c4c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:08.187020 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bae9a-a97e-4bee-9487-d586b98c4c91 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:08.191441 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb93166-825c-4cde-bcec-794aade10f6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:08.192680 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb93166-825c-4cde-bcec-794aade10f6c service cinder] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:08.192970 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb93166-825c-4cde-bcec-794aade10f6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:08.193248 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb93166-825c-4cde-bcec-794aade10f6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:08.251542 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:08.254646 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb93166-825c-4cde-bcec-794aade10f6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:08.257251 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb93166-825c-4cde-bcec-794aade10f6c service cinder] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:08.267656 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1268/2530] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:20:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:20:12.690845 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca6591-bb65-4d63-82e5-fd4dc074f3ca None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:12.692474 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-86ca6591-bb65-4d63-82e5-fd4dc074f3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:12.704989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca6591-bb65-4d63-82e5-fd4dc074f3ca None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:12.712503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde9fac-6a82-411c-9a01-7d92065ad1f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:12.713245 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde9fac-6a82-411c-9a01-7d92065ad1f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:12.713585 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde9fac-6a82-411c-9a01-7d92065ad1f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:12.713952 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde9fac-6a82-411c-9a01-7d92065ad1f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:12.726576 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde9fac-6a82-411c-9a01-7d92065ad1f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:12.728308 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde9fac-6a82-411c-9a01-7d92065ad1f7 service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:12.743023 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1263/2531] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:20:12 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:20:12.755520 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb93166-825c-4cde-bcec-794aade10f6c service cinder] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:12.757068 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-1eb93166-825c-4cde-bcec-794aade10f6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:12.864953 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb93166-825c-4cde-bcec-794aade10f6c service cinder] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:12.871560 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f02cea-d569-4d4e-9e3d-704d47217f0b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cc2655cc32e4d63a7000ef41bcbcfea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:12.872266 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f02cea-d569-4d4e-9e3d-704d47217f0b service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:12.872620 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f02cea-d569-4d4e-9e3d-704d47217f0b service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:12.873034 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f02cea-d569-4d4e-9e3d-704d47217f0b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:12.875128 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-68f02cea-d569-4d4e-9e3d-704d47217f0b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:12.892026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f02cea-d569-4d4e-9e3d-704d47217f0b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:12.894341 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f02cea-d569-4d4e-9e3d-704d47217f0b service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:12.910992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1269/2532] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:20:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:20:17.739707 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde9fac-6a82-411c-9a01-7d92065ad1f7 service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:17.801707 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde9fac-6a82-411c-9a01-7d92065ad1f7 service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:17.806287 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40416c-c0d0-4c82-a972-61823292771c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:17.806933 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40416c-c0d0-4c82-a972-61823292771c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:17.807217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40416c-c0d0-4c82-a972-61823292771c admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:17.807490 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40416c-c0d0-4c82-a972-61823292771c admin admin] PATH_INFO: `/v3/users/45c529adbb514287999752cb54bba689` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:17.808612 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40416c-c0d0-4c82-a972-61823292771c admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c529adbb514287999752cb54bba689)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:17.810550 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40416c-c0d0-4c82-a972-61823292771c admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:17.856176 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4b40416c-c0d0-4c82-a972-61823292771c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c529adbb514287999752cb54bba689'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:17.862997 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4b40416c-c0d0-4c82-a972-61823292771c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c529adbb514287999752cb54bba689'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:17.863659 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4b40416c-c0d0-4c82-a972-61823292771c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c529adbb514287999752cb54bba689'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:17.871881 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4b40416c-c0d0-4c82-a972-61823292771c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c529adbb514287999752cb54bba689'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:17.884041 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4b40416c-c0d0-4c82-a972-61823292771c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c529adbb514287999752cb54bba689'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:17.889871 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1264/2533] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:20:17 2022] DELETE /identity/v3/users/45c529adbb514287999752cb54bba689 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:20:17.895227 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6b775-b06c-4459-8b99-0ae2337d6d83 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:17.895744 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6b775-b06c-4459-8b99-0ae2337d6d83 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:17.896268 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6b775-b06c-4459-8b99-0ae2337d6d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:17.915904 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-b9f6b775-b06c-4459-8b99-0ae2337d6d83 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:17.917115 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b9f6b775-b06c-4459-8b99-0ae2337d6d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:17.976381 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1270/2534] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:20:17.985906 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40416c-c0d0-4c82-a972-61823292771c admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:17.987387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4b40416c-c0d0-4c82-a972-61823292771c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:18.009611 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:18.047365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40416c-c0d0-4c82-a972-61823292771c admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:18.051376 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2d39d-ee7b-4918-985a-844f4a33186b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:18.051980 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2d39d-ee7b-4918-985a-844f4a33186b service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:18.052260 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2d39d-ee7b-4918-985a-844f4a33186b service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:18.052530 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2d39d-ee7b-4918-985a-844f4a33186b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:18.061391 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2d39d-ee7b-4918-985a-844f4a33186b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:18.062821 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2d39d-ee7b-4918-985a-844f4a33186b service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:18.073097 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1265/2535] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:20:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:20:18.345101 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6b775-b06c-4459-8b99-0ae2337d6d83 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:18.346677 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b9f6b775-b06c-4459-8b99-0ae2337d6d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:18.359160 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6b775-b06c-4459-8b99-0ae2337d6d83 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:18.364531 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669fefeb-dfc8-41ac-a8fb-b2d55134b681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:18.365187 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669fefeb-dfc8-41ac-a8fb-b2d55134b681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:18.365912 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669fefeb-dfc8-41ac-a8fb-b2d55134b681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:18.366198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669fefeb-dfc8-41ac-a8fb-b2d55134b681 admin admin] PATH_INFO: `/v3/projects/de3faeed7ce64fa89d85fadeafc9bdf5` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:18.367408 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669fefeb-dfc8-41ac-a8fb-b2d55134b681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de3faeed7ce64fa89d85fadeafc9bdf5)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:18.369321 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669fefeb-dfc8-41ac-a8fb-b2d55134b681 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:18.416745 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-669fefeb-dfc8-41ac-a8fb-b2d55134b681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de3faeed7ce64fa89d85fadeafc9bdf5', 'request_id': 'req-669fefeb-dfc8-41ac-a8fb-b2d55134b681'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:18.422312 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-669fefeb-dfc8-41ac-a8fb-b2d55134b681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de3faeed7ce64fa89d85fadeafc9bdf5', 'request_id': 'req-669fefeb-dfc8-41ac-a8fb-b2d55134b681'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:18.424108 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:18.430291 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-669fefeb-dfc8-41ac-a8fb-b2d55134b681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de3faeed7ce64fa89d85fadeafc9bdf5', 'request_id': 'req-669fefeb-dfc8-41ac-a8fb-b2d55134b681'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:18.443967 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-669fefeb-dfc8-41ac-a8fb-b2d55134b681 admin admin] The token cache is being invalidate because project de3faeed7ce64fa89d85fadeafc9bdf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:20:18.444262 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-669fefeb-dfc8-41ac-a8fb-b2d55134b681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:18.444899 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-669fefeb-dfc8-41ac-a8fb-b2d55134b681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:18.446500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1271/2536] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:20:18 2022] DELETE /identity/v3/projects/de3faeed7ce64fa89d85fadeafc9bdf5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:20:18.450461 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2d39d-ee7b-4918-985a-844f4a33186b service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:18.460151 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2d39d-ee7b-4918-985a-844f4a33186b service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:18.517204 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dd6f8-6a1e-4d92-a5d9-28565a2e616b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:18.517749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dd6f8-6a1e-4d92-a5d9-28565a2e616b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:18.518048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dd6f8-6a1e-4d92-a5d9-28565a2e616b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:18.518320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dd6f8-6a1e-4d92-a5d9-28565a2e616b admin admin] PATH_INFO: `/v3/users/82a60b540142469e948e381ce8af58f5` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:18.519614 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217dd6f8-6a1e-4d92-a5d9-28565a2e616b admin admin] RBAC: Authorizing `identity:delete_user(user_id=82a60b540142469e948e381ce8af58f5)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:18.521638 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217dd6f8-6a1e-4d92-a5d9-28565a2e616b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:18.565656 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-217dd6f8-6a1e-4d92-a5d9-28565a2e616b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82a60b540142469e948e381ce8af58f5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:18.572922 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-217dd6f8-6a1e-4d92-a5d9-28565a2e616b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82a60b540142469e948e381ce8af58f5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:18.573579 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-217dd6f8-6a1e-4d92-a5d9-28565a2e616b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82a60b540142469e948e381ce8af58f5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:18.580585 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-217dd6f8-6a1e-4d92-a5d9-28565a2e616b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82a60b540142469e948e381ce8af58f5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:18.593588 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-217dd6f8-6a1e-4d92-a5d9-28565a2e616b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82a60b540142469e948e381ce8af58f5'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:18.600583 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1266/2537] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:20:18 2022] DELETE /identity/v3/users/82a60b540142469e948e381ce8af58f5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:20:18.955087 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669fefeb-dfc8-41ac-a8fb-b2d55134b681 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:18.956754 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-669fefeb-dfc8-41ac-a8fb-b2d55134b681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:18.975054 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669fefeb-dfc8-41ac-a8fb-b2d55134b681 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:19.033916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e7f4d-34ae-4091-8456-10722d334617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:19.034470 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e7f4d-34ae-4091-8456-10722d334617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:19.034749 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e7f4d-34ae-4091-8456-10722d334617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:19.035080 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e7f4d-34ae-4091-8456-10722d334617 admin admin] PATH_INFO: `/v3/projects/ae618ca249b045e2af0981b22d243a38` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:19.036183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e7f4d-34ae-4091-8456-10722d334617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae618ca249b045e2af0981b22d243a38)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:19.038054 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e7f4d-34ae-4091-8456-10722d334617 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:19.090980 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4d9e7f4d-34ae-4091-8456-10722d334617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae618ca249b045e2af0981b22d243a38', 'request_id': 'req-4d9e7f4d-34ae-4091-8456-10722d334617'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:19.097988 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4d9e7f4d-34ae-4091-8456-10722d334617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae618ca249b045e2af0981b22d243a38', 'request_id': 'req-4d9e7f4d-34ae-4091-8456-10722d334617'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:19.104950 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4d9e7f4d-34ae-4091-8456-10722d334617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae618ca249b045e2af0981b22d243a38', 'request_id': 'req-4d9e7f4d-34ae-4091-8456-10722d334617'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:19.119513 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4d9e7f4d-34ae-4091-8456-10722d334617 admin admin] The token cache is being invalidate because project ae618ca249b045e2af0981b22d243a38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:20:19.119804 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4d9e7f4d-34ae-4091-8456-10722d334617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:19.120485 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4d9e7f4d-34ae-4091-8456-10722d334617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:19.122087 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1272/2538] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:20:18 2022] DELETE /identity/v3/projects/ae618ca249b045e2af0981b22d243a38 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:20:20.159543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61297e57-259a-4d14-90b8-f94c7f32ddcf None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:20.160076 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61297e57-259a-4d14-90b8-f94c7f32ddcf None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:20.160211 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61297e57-259a-4d14-90b8-f94c7f32ddcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:20.180272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-61297e57-259a-4d14-90b8-f94c7f32ddcf None None] MFA Rules not processed for user `4d30a0379df643ff81e1ca01f3e26d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:20.181460 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-61297e57-259a-4d14-90b8-f94c7f32ddcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:20.219836 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:20.239460 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1267/2539] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:20:28.923151 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa64e6-f8b0-40f6-9e71-9a9950f000bf None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:28.923614 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa64e6-f8b0-40f6-9e71-9a9950f000bf None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:28.923711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa64e6-f8b0-40f6-9e71-9a9950f000bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:28.952053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-9aaa64e6-f8b0-40f6-9e71-9a9950f000bf None None] MFA Rules not processed for user `280d47ed79f14105a83b18b2cac58b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:28.953239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-9aaa64e6-f8b0-40f6-9e71-9a9950f000bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:29.006375 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1273/2540] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Oct 17 03:20:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:20:29.047755 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61297e57-259a-4d14-90b8-f94c7f32ddcf None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:29.049407 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-61297e57-259a-4d14-90b8-f94c7f32ddcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:29.119192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61297e57-259a-4d14-90b8-f94c7f32ddcf None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:29.124019 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbb090-f0f7-4473-91c0-4cc5f313eb79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eceda8b3a8ba4233851d0138d8f4d06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:29.124628 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbb090-f0f7-4473-91c0-4cc5f313eb79 service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:29.124915 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbb090-f0f7-4473-91c0-4cc5f313eb79 service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:29.125196 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbb090-f0f7-4473-91c0-4cc5f313eb79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:29.133329 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cbb090-f0f7-4473-91c0-4cc5f313eb79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:29.135022 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cbb090-f0f7-4473-91c0-4cc5f313eb79 service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:29.143242 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1268/2541] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Oct 17 03:20:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 17 03:20:29.653681 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ca2bb-acb1-4945-bf24-148e09028b61 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:29.654046 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ca2bb-acb1-4945-bf24-148e09028b61 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:29.654384 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ca2bb-acb1-4945-bf24-148e09028b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:29.674071 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-414ca2bb-acb1-4945-bf24-148e09028b61 None None] MFA Rules not processed for user `280d47ed79f14105a83b18b2cac58b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:29.675296 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-414ca2bb-acb1-4945-bf24-148e09028b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:29.682260 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1274/2542] 10.222.0.27 () {58 vars in 967 bytes} [Mon Oct 17 03:20:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:20:29.694239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbb090-f0f7-4473-91c0-4cc5f313eb79 service swift] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:29.705083 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbb090-f0f7-4473-91c0-4cc5f313eb79 service swift] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:29.711267 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2caa12-7ebb-4f24-a6b2-9cd4cf759883 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eceda8b3a8ba4233851d0138d8f4d06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:29.711906 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2caa12-7ebb-4f24-a6b2-9cd4cf759883 service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:29.712228 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2caa12-7ebb-4f24-a6b2-9cd4cf759883 service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:29.712525 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2caa12-7ebb-4f24-a6b2-9cd4cf759883 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:29.720501 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2caa12-7ebb-4f24-a6b2-9cd4cf759883 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:29.722071 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2caa12-7ebb-4f24-a6b2-9cd4cf759883 service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:29.730672 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1269/2543] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Oct 17 03:20:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:20:29.961972 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675034a-33d2-42ca-8d56-9d00fa611a62 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:29.962449 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675034a-33d2-42ca-8d56-9d00fa611a62 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:29.962564 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675034a-33d2-42ca-8d56-9d00fa611a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:29.981992 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-a675034a-33d2-42ca-8d56-9d00fa611a62 None None] MFA Rules not processed for user `280d47ed79f14105a83b18b2cac58b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:29.983214 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-a675034a-33d2-42ca-8d56-9d00fa611a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:29.991830 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1275/2544] 10.222.0.27 () {58 vars in 1009 bytes} [Mon Oct 17 03:20:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:20:30.003598 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2caa12-7ebb-4f24-a6b2-9cd4cf759883 service swift] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:30.014824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2caa12-7ebb-4f24-a6b2-9cd4cf759883 service swift] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:30.021591 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdfd5c-1615-4c5f-8483-388039777dad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eceda8b3a8ba4233851d0138d8f4d06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:30.022185 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdfd5c-1615-4c5f-8483-388039777dad service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:30.022468 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdfd5c-1615-4c5f-8483-388039777dad service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:30.022742 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdfd5c-1615-4c5f-8483-388039777dad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:30.030799 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdfd5c-1615-4c5f-8483-388039777dad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:30.032413 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdfd5c-1615-4c5f-8483-388039777dad service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:30.041821 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1270/2545] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Oct 17 03:20:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:20:30.361183 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168e8c0-2dd0-407c-bed7-90e50b241a8b None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:30.361476 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168e8c0-2dd0-407c-bed7-90e50b241a8b None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:30.361759 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168e8c0-2dd0-407c-bed7-90e50b241a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:30.369895 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:30.384423 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-c168e8c0-2dd0-407c-bed7-90e50b241a8b None None] MFA Rules not processed for user `280d47ed79f14105a83b18b2cac58b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:30.385673 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c168e8c0-2dd0-407c-bed7-90e50b241a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:30.392760 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1276/2546] 10.222.0.27 () {58 vars in 967 bytes} [Mon Oct 17 03:20:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:20:30.405352 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdfd5c-1615-4c5f-8483-388039777dad service swift] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:30.416716 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdfd5c-1615-4c5f-8483-388039777dad service swift] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:30.421589 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09a5f5-e5f5-4d93-a178-3eebda9109eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eceda8b3a8ba4233851d0138d8f4d06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:30.422229 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09a5f5-e5f5-4d93-a178-3eebda9109eb service swift] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:30.422455 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09a5f5-e5f5-4d93-a178-3eebda9109eb service swift] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:30.422729 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09a5f5-e5f5-4d93-a178-3eebda9109eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:30.430664 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d09a5f5-e5f5-4d93-a178-3eebda9109eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:30.432272 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d09a5f5-e5f5-4d93-a178-3eebda9109eb service swift] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:30.440144 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1271/2547] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Oct 17 03:20:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:20:32.759915 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168e8c0-2dd0-407c-bed7-90e50b241a8b None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:32.761319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c168e8c0-2dd0-407c-bed7-90e50b241a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:32.826165 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168e8c0-2dd0-407c-bed7-90e50b241a8b None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:32.830591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0885e-1418-4654-91d4-d036e812fb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:32.831241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0885e-1418-4654-91d4-d036e812fb90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:32.831520 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0885e-1418-4654-91d4-d036e812fb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:32.831789 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0885e-1418-4654-91d4-d036e812fb90 admin admin] PATH_INFO: `/v3/users/1ba9dfbbcb5c424da17e7b198aef35d9` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:32.833402 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0885e-1418-4654-91d4-d036e812fb90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ba9dfbbcb5c424da17e7b198aef35d9)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:32.835336 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0885e-1418-4654-91d4-d036e812fb90 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:32.880972 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-abc0885e-1418-4654-91d4-d036e812fb90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ba9dfbbcb5c424da17e7b198aef35d9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:32.887647 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-abc0885e-1418-4654-91d4-d036e812fb90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ba9dfbbcb5c424da17e7b198aef35d9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:32.888290 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-abc0885e-1418-4654-91d4-d036e812fb90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ba9dfbbcb5c424da17e7b198aef35d9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:32.896198 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-abc0885e-1418-4654-91d4-d036e812fb90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ba9dfbbcb5c424da17e7b198aef35d9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:32.907679 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-abc0885e-1418-4654-91d4-d036e812fb90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ba9dfbbcb5c424da17e7b198aef35d9'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:32.914271 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1277/2548] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:20:32 2022] DELETE /identity/v3/users/1ba9dfbbcb5c424da17e7b198aef35d9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:20:32.918797 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcdce1-8b19-4f1b-862a-56672ab0016a None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:32.919335 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcdce1-8b19-4f1b-862a-56672ab0016a None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:32.919461 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcdce1-8b19-4f1b-862a-56672ab0016a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:32.938207 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-7ffcdce1-8b19-4f1b-862a-56672ab0016a None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:20:32.939437 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7ffcdce1-8b19-4f1b-862a-56672ab0016a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:32.997041 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1272/2549] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:20:33.007645 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0885e-1418-4654-91d4-d036e812fb90 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:33.009055 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-abc0885e-1418-4654-91d4-d036e812fb90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:33.066497 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0885e-1418-4654-91d4-d036e812fb90 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:33.070468 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8ab4c-9eb8-41df-a195-365a904ac107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:33.071091 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8ab4c-9eb8-41df-a195-365a904ac107 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:33.071370 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8ab4c-9eb8-41df-a195-365a904ac107 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:33.071642 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8ab4c-9eb8-41df-a195-365a904ac107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:33.080372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df8ab4c-9eb8-41df-a195-365a904ac107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:33.081785 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df8ab4c-9eb8-41df-a195-365a904ac107 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:33.091244 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1278/2550] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:20:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:20:33.259629 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcdce1-8b19-4f1b-862a-56672ab0016a None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:33.261076 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7ffcdce1-8b19-4f1b-862a-56672ab0016a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:33.275213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcdce1-8b19-4f1b-862a-56672ab0016a None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:33.288055 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce732a94-de99-4600-a4a5-bff5dbc041e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:33.288753 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce732a94-de99-4600-a4a5-bff5dbc041e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:33.289293 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce732a94-de99-4600-a4a5-bff5dbc041e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:33.289749 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce732a94-de99-4600-a4a5-bff5dbc041e8 admin admin] PATH_INFO: `/v3/projects/b681923b7d7b4cf682ae41fdce65a823` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:33.291004 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce732a94-de99-4600-a4a5-bff5dbc041e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b681923b7d7b4cf682ae41fdce65a823)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:33.292913 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce732a94-de99-4600-a4a5-bff5dbc041e8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:33.330449 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:33.343734 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ce732a94-de99-4600-a4a5-bff5dbc041e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b681923b7d7b4cf682ae41fdce65a823', 'request_id': 'req-ce732a94-de99-4600-a4a5-bff5dbc041e8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:33.349921 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ce732a94-de99-4600-a4a5-bff5dbc041e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b681923b7d7b4cf682ae41fdce65a823', 'request_id': 'req-ce732a94-de99-4600-a4a5-bff5dbc041e8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:33.356213 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ce732a94-de99-4600-a4a5-bff5dbc041e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b681923b7d7b4cf682ae41fdce65a823', 'request_id': 'req-ce732a94-de99-4600-a4a5-bff5dbc041e8'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:33.368479 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ce732a94-de99-4600-a4a5-bff5dbc041e8 admin admin] The token cache is being invalidate because project b681923b7d7b4cf682ae41fdce65a823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:20:33.368767 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ce732a94-de99-4600-a4a5-bff5dbc041e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:33.369384 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-ce732a94-de99-4600-a4a5-bff5dbc041e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:20:33.370854 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1273/2551] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:20:33 2022] DELETE /identity/v3/projects/b681923b7d7b4cf682ae41fdce65a823 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:20:38.354716 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8ab4c-9eb8-41df-a195-365a904ac107 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:20:38.356156 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4df8ab4c-9eb8-41df-a195-365a904ac107 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:38.367550 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8ab4c-9eb8-41df-a195-365a904ac107 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:20:38.379952 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:38.418600 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fc816-ac10-446e-a917-70c9a7613979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:20:38.419238 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fc816-ac10-446e-a917-70c9a7613979 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:20:38.419522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fc816-ac10-446e-a917-70c9a7613979 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:20:38.419797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fc816-ac10-446e-a917-70c9a7613979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:20:38.421509 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c52fc816-ac10-446e-a917-70c9a7613979 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:20:38.484360 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52fc816-ac10-446e-a917-70c9a7613979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:20:38.485691 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52fc816-ac10-446e-a917-70c9a7613979 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:20:38.495112 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1279/2552] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:20:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:20:43.365494 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:20:48.490918 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=42 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:21:09.858422 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43015f-75e5-4153-b3ff-481ad3011081 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:09.858900 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43015f-75e5-4153-b3ff-481ad3011081 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:09.859048 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43015f-75e5-4153-b3ff-481ad3011081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:09.878835 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-8e43015f-75e5-4153-b3ff-481ad3011081 None None] MFA Rules not processed for user `280d47ed79f14105a83b18b2cac58b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:21:09.881094 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8e43015f-75e5-4153-b3ff-481ad3011081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:21:09.936317 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1274/2553] 10.222.0.27 () {58 vars in 967 bytes} [Mon Oct 17 03:21:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:21:09.949372 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fc816-ac10-446e-a917-70c9a7613979 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:09.950729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c52fc816-ac10-446e-a917-70c9a7613979 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:21:10.028650 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fc816-ac10-446e-a917-70c9a7613979 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:10.033537 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc689b18-1bdf-46b8-8d05-2de954191ce8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eceda8b3a8ba4233851d0138d8f4d06a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:10.034146 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc689b18-1bdf-46b8-8d05-2de954191ce8 service swift] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:10.034429 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc689b18-1bdf-46b8-8d05-2de954191ce8 service swift] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:10.034707 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc689b18-1bdf-46b8-8d05-2de954191ce8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:10.043774 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc689b18-1bdf-46b8-8d05-2de954191ce8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:10.045371 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc689b18-1bdf-46b8-8d05-2de954191ce8 service swift] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:10.053382 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1280/2554] 10.222.0.27 () {60 vars in 1384 bytes} [Mon Oct 17 03:21:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:21:16.452596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43015f-75e5-4153-b3ff-481ad3011081 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:16.458945 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-8e43015f-75e5-4153-b3ff-481ad3011081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:21:16.532924 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43015f-75e5-4153-b3ff-481ad3011081 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:16.537655 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70533cf5-5e7c-445b-abe1-ee8776ee1674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:16.538281 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70533cf5-5e7c-445b-abe1-ee8776ee1674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:16.538561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70533cf5-5e7c-445b-abe1-ee8776ee1674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:16.538831 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70533cf5-5e7c-445b-abe1-ee8776ee1674 admin admin] PATH_INFO: `/v3/users/82da4b8f8bdb4dbab655ff510bfa328f` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:16.539993 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70533cf5-5e7c-445b-abe1-ee8776ee1674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82da4b8f8bdb4dbab655ff510bfa328f)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:16.542187 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70533cf5-5e7c-445b-abe1-ee8776ee1674 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:16.587041 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-70533cf5-5e7c-445b-abe1-ee8776ee1674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82da4b8f8bdb4dbab655ff510bfa328f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:16.593824 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-70533cf5-5e7c-445b-abe1-ee8776ee1674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82da4b8f8bdb4dbab655ff510bfa328f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:16.594491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-70533cf5-5e7c-445b-abe1-ee8776ee1674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82da4b8f8bdb4dbab655ff510bfa328f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:16.601497 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-70533cf5-5e7c-445b-abe1-ee8776ee1674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82da4b8f8bdb4dbab655ff510bfa328f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:16.613259 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-70533cf5-5e7c-445b-abe1-ee8776ee1674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82da4b8f8bdb4dbab655ff510bfa328f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:16.619690 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1275/2555] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:21:16 2022] DELETE /identity/v3/users/82da4b8f8bdb4dbab655ff510bfa328f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:21:16.790529 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc689b18-1bdf-46b8-8d05-2de954191ce8 service swift] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:16.810711 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc689b18-1bdf-46b8-8d05-2de954191ce8 service swift] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:16.868008 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e406b4-8896-4bcf-9c56-c2a6774760f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:16.868569 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e406b4-8896-4bcf-9c56-c2a6774760f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:16.868849 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e406b4-8896-4bcf-9c56-c2a6774760f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:16.869121 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e406b4-8896-4bcf-9c56-c2a6774760f5 admin admin] PATH_INFO: `/v3/projects/dc518d9b31644a8d8b21cf9c645b1373` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:16.870226 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e406b4-8896-4bcf-9c56-c2a6774760f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc518d9b31644a8d8b21cf9c645b1373)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:16.872144 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e406b4-8896-4bcf-9c56-c2a6774760f5 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:16.917032 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b0e406b4-8896-4bcf-9c56-c2a6774760f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc518d9b31644a8d8b21cf9c645b1373', 'request_id': 'req-b0e406b4-8896-4bcf-9c56-c2a6774760f5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:16.922496 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b0e406b4-8896-4bcf-9c56-c2a6774760f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc518d9b31644a8d8b21cf9c645b1373', 'request_id': 'req-b0e406b4-8896-4bcf-9c56-c2a6774760f5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:16.932881 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b0e406b4-8896-4bcf-9c56-c2a6774760f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc518d9b31644a8d8b21cf9c645b1373', 'request_id': 'req-b0e406b4-8896-4bcf-9c56-c2a6774760f5'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:16.945909 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b0e406b4-8896-4bcf-9c56-c2a6774760f5 admin admin] The token cache is being invalidate because project dc518d9b31644a8d8b21cf9c645b1373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:21:16.946202 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b0e406b4-8896-4bcf-9c56-c2a6774760f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:16.946890 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-b0e406b4-8896-4bcf-9c56-c2a6774760f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:16.948445 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1281/2556] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:21:16 2022] DELETE /identity/v3/projects/dc518d9b31644a8d8b21cf9c645b1373 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:21:16.952342 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70533cf5-5e7c-445b-abe1-ee8776ee1674 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:16.953775 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-70533cf5-5e7c-445b-abe1-ee8776ee1674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:21:16.967241 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70533cf5-5e7c-445b-abe1-ee8776ee1674 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:16.986817 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:21:17.025875 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cf7fd-37eb-49ff-8761-c14af0d4610f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:17.026497 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cf7fd-37eb-49ff-8761-c14af0d4610f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:17.026772 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cf7fd-37eb-49ff-8761-c14af0d4610f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:17.027070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cf7fd-37eb-49ff-8761-c14af0d4610f admin admin] PATH_INFO: `/v3/users/c85ec9988c5b4002bab7d2267a305593` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:17.028187 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cf7fd-37eb-49ff-8761-c14af0d4610f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c85ec9988c5b4002bab7d2267a305593)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:17.030089 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cf7fd-37eb-49ff-8761-c14af0d4610f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:17.075294 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c96cf7fd-37eb-49ff-8761-c14af0d4610f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c85ec9988c5b4002bab7d2267a305593'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:17.082055 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c96cf7fd-37eb-49ff-8761-c14af0d4610f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c85ec9988c5b4002bab7d2267a305593'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:17.082722 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c96cf7fd-37eb-49ff-8761-c14af0d4610f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c85ec9988c5b4002bab7d2267a305593'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:17.090164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c96cf7fd-37eb-49ff-8761-c14af0d4610f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c85ec9988c5b4002bab7d2267a305593'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:17.102380 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c96cf7fd-37eb-49ff-8761-c14af0d4610f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c85ec9988c5b4002bab7d2267a305593'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:17.108061 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1276/2557] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:21:16 2022] DELETE /identity/v3/users/c85ec9988c5b4002bab7d2267a305593 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:21:17.280846 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e406b4-8896-4bcf-9c56-c2a6774760f5 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:17.282354 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-b0e406b4-8896-4bcf-9c56-c2a6774760f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:21:17.296940 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e406b4-8896-4bcf-9c56-c2a6774760f5 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:17.316729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:21:17.359415 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8d5ae-5b47-430f-8f96-20151e8ef46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:17.360060 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8d5ae-5b47-430f-8f96-20151e8ef46a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:17.360344 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8d5ae-5b47-430f-8f96-20151e8ef46a admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:17.360622 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8d5ae-5b47-430f-8f96-20151e8ef46a admin admin] PATH_INFO: `/v3/projects/83ef05f3398644eda7a2932e00314109` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:17.361857 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf8d5ae-5b47-430f-8f96-20151e8ef46a admin admin] RBAC: Authorizing `identity:delete_project(project_id=83ef05f3398644eda7a2932e00314109)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:17.364053 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf8d5ae-5b47-430f-8f96-20151e8ef46a admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:17.410616 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dcf8d5ae-5b47-430f-8f96-20151e8ef46a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83ef05f3398644eda7a2932e00314109', 'request_id': 'req-dcf8d5ae-5b47-430f-8f96-20151e8ef46a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:17.417260 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dcf8d5ae-5b47-430f-8f96-20151e8ef46a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83ef05f3398644eda7a2932e00314109', 'request_id': 'req-dcf8d5ae-5b47-430f-8f96-20151e8ef46a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:17.423379 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dcf8d5ae-5b47-430f-8f96-20151e8ef46a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83ef05f3398644eda7a2932e00314109', 'request_id': 'req-dcf8d5ae-5b47-430f-8f96-20151e8ef46a'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:17.435322 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dcf8d5ae-5b47-430f-8f96-20151e8ef46a admin admin] The token cache is being invalidate because project 83ef05f3398644eda7a2932e00314109 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:21:17.435595 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dcf8d5ae-5b47-430f-8f96-20151e8ef46a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:17.436222 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-dcf8d5ae-5b47-430f-8f96-20151e8ef46a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:21:17.437675 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1282/2558] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:21:17 2022] DELETE /identity/v3/projects/83ef05f3398644eda7a2932e00314109 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:21:17.443497 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493b9f1-a225-4f25-aae5-9dd70511b5c9 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:17.443895 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493b9f1-a225-4f25-aae5-9dd70511b5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:17.444245 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493b9f1-a225-4f25-aae5-9dd70511b5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:17.465990 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-7493b9f1-a225-4f25-aae5-9dd70511b5c9 None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:21:17.467259 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-7493b9f1-a225-4f25-aae5-9dd70511b5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:21:17.529906 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1277/2559] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:21:17.539209 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0539ece-1bd6-47a4-902c-a8ac1e9cbb6d None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:17.539708 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0539ece-1bd6-47a4-902c-a8ac1e9cbb6d None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:17.539989 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0539ece-1bd6-47a4-902c-a8ac1e9cbb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:17.560624 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-e0539ece-1bd6-47a4-902c-a8ac1e9cbb6d None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:21:17.561828 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-e0539ece-1bd6-47a4-902c-a8ac1e9cbb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:21:17.569920 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1283/2560] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:21:17.574172 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493b9f1-a225-4f25-aae5-9dd70511b5c9 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:17.584499 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493b9f1-a225-4f25-aae5-9dd70511b5c9 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:17.589181 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee2aa8-d91d-430f-b0fe-eb1d9cdd734d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:17.589787 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee2aa8-d91d-430f-b0fe-eb1d9cdd734d admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:17.590066 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee2aa8-d91d-430f-b0fe-eb1d9cdd734d admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:17.590338 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee2aa8-d91d-430f-b0fe-eb1d9cdd734d admin admin] PATH_INFO: `/v3/domains` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:17.591029 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee2aa8-d91d-430f-b0fe-eb1d9cdd734d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:21:17.591337 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee2aa8-d91d-430f-b0fe-eb1d9cdd734d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:17.592938 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee2aa8-d91d-430f-b0fe-eb1d9cdd734d admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:17.605230 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1278/2561] 10.222.0.27 () {58 vars in 1153 bytes} [Mon Oct 17 03:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:21:17.609365 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0539ece-1bd6-47a4-902c-a8ac1e9cbb6d None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:17.618752 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0539ece-1bd6-47a4-902c-a8ac1e9cbb6d None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:17.623350 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55469191-61c5-4df0-99a7-532315d0163c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:17.623939 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55469191-61c5-4df0-99a7-532315d0163c admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:17.624214 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55469191-61c5-4df0-99a7-532315d0163c admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:17.624480 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55469191-61c5-4df0-99a7-532315d0163c admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:17.625098 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55469191-61c5-4df0-99a7-532315d0163c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:17.626997 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55469191-61c5-4df0-99a7-532315d0163c admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:17.654109 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1284/2562] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:21:17 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:21:17.658446 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee2aa8-d91d-430f-b0fe-eb1d9cdd734d admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:17.669758 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee2aa8-d91d-430f-b0fe-eb1d9cdd734d admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:17.698304 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:21:17.727710 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bca676-b476-4a26-992a-09d5a04b2f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:17.728320 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bca676-b476-4a26-992a-09d5a04b2f0b admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:17.728602 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bca676-b476-4a26-992a-09d5a04b2f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:17.728877 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bca676-b476-4a26-992a-09d5a04b2f0b admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:17.729503 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bca676-b476-4a26-992a-09d5a04b2f0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:17.731677 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bca676-b476-4a26-992a-09d5a04b2f0b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:17.752332 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1279/2563] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:21:17 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:21:17.757436 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55469191-61c5-4df0-99a7-532315d0163c admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:17.767628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55469191-61c5-4df0-99a7-532315d0163c admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:17.773021 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095ea47-4b70-4b5d-8fd6-8777ab7abe3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:17.773619 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095ea47-4b70-4b5d-8fd6-8777ab7abe3e admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:17.773902 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095ea47-4b70-4b5d-8fd6-8777ab7abe3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:17.774178 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095ea47-4b70-4b5d-8fd6-8777ab7abe3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:17.774818 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095ea47-4b70-4b5d-8fd6-8777ab7abe3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:17.776439 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095ea47-4b70-4b5d-8fd6-8777ab7abe3e admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:17.786364 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1285/2564] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:21:17.790440 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bca676-b476-4a26-992a-09d5a04b2f0b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:17.801139 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bca676-b476-4a26-992a-09d5a04b2f0b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:17.806208 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13ec37-4867-4d88-a063-aa6b763655b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:17.806822 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13ec37-4867-4d88-a063-aa6b763655b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:17.807140 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13ec37-4867-4d88-a063-aa6b763655b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:17.807408 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13ec37-4867-4d88-a063-aa6b763655b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:17.808073 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b13ec37-4867-4d88-a063-aa6b763655b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:17.809685 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b13ec37-4867-4d88-a063-aa6b763655b8 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:17.820707 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1280/2565] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:21:17.825009 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095ea47-4b70-4b5d-8fd6-8777ab7abe3e admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:17.835342 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095ea47-4b70-4b5d-8fd6-8777ab7abe3e admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:17.839874 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238de4f5-faaf-479b-b733-fe4421a5e858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:17.840512 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238de4f5-faaf-479b-b733-fe4421a5e858 admin admin] REQUEST_METHOD: `PUT` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:17.840790 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238de4f5-faaf-479b-b733-fe4421a5e858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:17.841060 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238de4f5-faaf-479b-b733-fe4421a5e858 admin admin] PATH_INFO: `/v3/projects/f7a92b100ac14d44aff5e4083a49f5ad/users/2969d23a676c4b7fb6c7b611c5999fa4/roles/11de9783bae3450ea53955de9ebcc9d1` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:17.855100 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238de4f5-faaf-479b-b733-fe4421a5e858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7a92b100ac14d44aff5e4083a49f5ad, user_id=2969d23a676c4b7fb6c7b611c5999fa4, role_id=11de9783bae3450ea53955de9ebcc9d1)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:17.857713 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238de4f5-faaf-479b-b733-fe4421a5e858 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:17.870304 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1286/2566] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:21:17 2022] PUT /identity/v3/projects/f7a92b100ac14d44aff5e4083a49f5ad/users/2969d23a676c4b7fb6c7b611c5999fa4/roles/11de9783bae3450ea53955de9ebcc9d1 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:21:17.875239 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24941470-42ca-4571-82b4-d10715d8216c None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:17.875523 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24941470-42ca-4571-82b4-d10715d8216c None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:17.875795 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24941470-42ca-4571-82b4-d10715d8216c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:17.894653 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-24941470-42ca-4571-82b4-d10715d8216c None None] MFA Rules not processed for user `f84d693b9d0643c69861a5541952b43b`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:21:17.896578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-24941470-42ca-4571-82b4-d10715d8216c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:21:17.953192 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1281/2567] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:21:17.964071 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238de4f5-faaf-479b-b733-fe4421a5e858 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:17.965833 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-238de4f5-faaf-479b-b733-fe4421a5e858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:21:18.016762 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:21:18.024871 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238de4f5-faaf-479b-b733-fe4421a5e858 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:18.030234 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc53b7f-5234-407b-8a72-87dc6d6820e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:18.030844 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc53b7f-5234-407b-8a72-87dc6d6820e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:18.031148 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc53b7f-5234-407b-8a72-87dc6d6820e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:18.031455 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc53b7f-5234-407b-8a72-87dc6d6820e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:18.041014 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc53b7f-5234-407b-8a72-87dc6d6820e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:18.042455 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc53b7f-5234-407b-8a72-87dc6d6820e5 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:18.052938 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1287/2568] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:21:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:21:22.005570 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c7fb4-b836-4a23-8f47-aa40f7d85455 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:22.006081 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c7fb4-b836-4a23-8f47-aa40f7d85455 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:22.006202 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c7fb4-b836-4a23-8f47-aa40f7d85455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:22.041753 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-851c7fb4-b836-4a23-8f47-aa40f7d85455 None None] MFA Rules not processed for user `2969d23a676c4b7fb6c7b611c5999fa4`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:21:22.043151 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-851c7fb4-b836-4a23-8f47-aa40f7d85455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:21:22.115359 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1282/2569] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:21:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:21:22.120483 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc53b7f-5234-407b-8a72-87dc6d6820e5 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:22.131177 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc53b7f-5234-407b-8a72-87dc6d6820e5 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:22.135988 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16823d-0e93-4bfd-9c49-27532eb0ffc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:22.136613 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16823d-0e93-4bfd-9c49-27532eb0ffc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:22.137081 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16823d-0e93-4bfd-9c49-27532eb0ffc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:22.137366 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16823d-0e93-4bfd-9c49-27532eb0ffc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:22.138014 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16823d-0e93-4bfd-9c49-27532eb0ffc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:22.139983 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16823d-0e93-4bfd-9c49-27532eb0ffc1 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:22.169004 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1288/2570] 10.222.0.27 () {60 vars in 1153 bytes} [Mon Oct 17 03:21:22 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:21:22.173217 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c7fb4-b836-4a23-8f47-aa40f7d85455 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:22.185363 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c7fb4-b836-4a23-8f47-aa40f7d85455 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:22.228869 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:21:22.249644 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb432912-cca4-4acb-a878-5caeeea9f50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:22.250311 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb432912-cca4-4acb-a878-5caeeea9f50a admin admin] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:22.250591 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb432912-cca4-4acb-a878-5caeeea9f50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:22.250880 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb432912-cca4-4acb-a878-5caeeea9f50a admin admin] PATH_INFO: `/v3/users` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:22.251525 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb432912-cca4-4acb-a878-5caeeea9f50a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:22.253512 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb432912-cca4-4acb-a878-5caeeea9f50a admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:22.278668 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1283/2571] 10.222.0.27 () {60 vars in 1144 bytes} [Mon Oct 17 03:21:22 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:21:22.281043 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16823d-0e93-4bfd-9c49-27532eb0ffc1 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:22.292212 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16823d-0e93-4bfd-9c49-27532eb0ffc1 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:22.297255 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e1c28-c7b3-4ed8-9622-b4cdeb66b2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:22.297875 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e1c28-c7b3-4ed8-9622-b4cdeb66b2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:22.298156 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e1c28-c7b3-4ed8-9622-b4cdeb66b2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:22.298430 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019e1c28-c7b3-4ed8-9622-b4cdeb66b2b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:22.299106 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e1c28-c7b3-4ed8-9622-b4cdeb66b2b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:22.300712 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019e1c28-c7b3-4ed8-9622-b4cdeb66b2b6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:22.313139 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1289/2572] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:21:22.317410 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb432912-cca4-4acb-a878-5caeeea9f50a admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:22.327804 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb432912-cca4-4acb-a878-5caeeea9f50a admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:22.332365 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152421c-97d7-40c3-b269-31b14aab9936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:22.333051 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152421c-97d7-40c3-b269-31b14aab9936 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:22.333430 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152421c-97d7-40c3-b269-31b14aab9936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:22.333700 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152421c-97d7-40c3-b269-31b14aab9936 admin admin] PATH_INFO: `/v3/projects/53e87d643123424b964b82af7aaec9ba/users/c8ca1a9883fd4b0f9cb82e54fcc89e50/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:22.349188 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152421c-97d7-40c3-b269-31b14aab9936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e87d643123424b964b82af7aaec9ba, user_id=c8ca1a9883fd4b0f9cb82e54fcc89e50, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:22.351872 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152421c-97d7-40c3-b269-31b14aab9936 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:22.369657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1284/2573] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:21:22 2022] PUT /identity/v3/projects/53e87d643123424b964b82af7aaec9ba/users/c8ca1a9883fd4b0f9cb82e54fcc89e50/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:21:22.371152 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e1c28-c7b3-4ed8-9622-b4cdeb66b2b6 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:22.381738 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019e1c28-c7b3-4ed8-9622-b4cdeb66b2b6 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:22.439160 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f88630c-110b-4405-ba06-0f533243f1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:22.439791 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88630c-110b-4405-ba06-0f533243f1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:22.440051 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88630c-110b-4405-ba06-0f533243f1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:22.440324 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f88630c-110b-4405-ba06-0f533243f1d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:22.440954 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f88630c-110b-4405-ba06-0f533243f1d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:22.442603 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f88630c-110b-4405-ba06-0f533243f1d6 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:22.452278 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1290/2574] 10.222.0.27 () {58 vars in 1122 bytes} [Mon Oct 17 03:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:21:22.456525 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152421c-97d7-40c3-b269-31b14aab9936 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:22.467042 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152421c-97d7-40c3-b269-31b14aab9936 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:22.472527 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b785c-678e-4dad-88f8-4b8a2455b253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:22.473229 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b785c-678e-4dad-88f8-4b8a2455b253 admin admin] REQUEST_METHOD: `PUT` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:22.474233 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b785c-678e-4dad-88f8-4b8a2455b253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:22.474590 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b785c-678e-4dad-88f8-4b8a2455b253 admin admin] PATH_INFO: `/v3/projects/53e87d643123424b964b82af7aaec9ba/users/c8ca1a9883fd4b0f9cb82e54fcc89e50/roles/ea0fd0c10f2f4e529270edbe3525d4c6` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:22.476555 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b785c-678e-4dad-88f8-4b8a2455b253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e87d643123424b964b82af7aaec9ba, user_id=c8ca1a9883fd4b0f9cb82e54fcc89e50, role_id=ea0fd0c10f2f4e529270edbe3525d4c6)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:22.479229 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b785c-678e-4dad-88f8-4b8a2455b253 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:22.490340 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1285/2575] 10.222.0.27 () {60 vars in 1483 bytes} [Mon Oct 17 03:21:22 2022] PUT /identity/v3/projects/53e87d643123424b964b82af7aaec9ba/users/c8ca1a9883fd4b0f9cb82e54fcc89e50/roles/ea0fd0c10f2f4e529270edbe3525d4c6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:21:26.972308 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c1415-7ab1-431f-ad8c-2b91cb53eadb None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:26.973389 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c1415-7ab1-431f-ad8c-2b91cb53eadb None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:26.974333 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c1415-7ab1-431f-ad8c-2b91cb53eadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:27.015454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-fa2c1415-7ab1-431f-ad8c-2b91cb53eadb None None] MFA Rules not processed for user `c8ca1a9883fd4b0f9cb82e54fcc89e50`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:21:27.017122 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-fa2c1415-7ab1-431f-ad8c-2b91cb53eadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:21:27.103855 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1291/2576] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:21:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:21:27.121279 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b785c-678e-4dad-88f8-4b8a2455b253 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:27.123617 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-233b785c-678e-4dad-88f8-4b8a2455b253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:21:27.225434 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b785c-678e-4dad-88f8-4b8a2455b253 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:27.233903 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e6b16-d6c5-4cea-9e33-9aad19dd3fd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:27.234583 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e6b16-d6c5-4cea-9e33-9aad19dd3fd1 service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:27.234858 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e6b16-d6c5-4cea-9e33-9aad19dd3fd1 service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:27.235640 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e6b16-d6c5-4cea-9e33-9aad19dd3fd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:27.301328 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e6b16-d6c5-4cea-9e33-9aad19dd3fd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:27.303052 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e6b16-d6c5-4cea-9e33-9aad19dd3fd1 service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:27.313491 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1286/2577] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:21:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:21:28.406597 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed34136-3f19-428b-bf19-6423b47849c7 None None] REQUEST_METHOD: `POST` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:28.407107 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed34136-3f19-428b-bf19-6423b47849c7 None None] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:28.407247 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed34136-3f19-428b-bf19-6423b47849c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:28.428323 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.auth.core [None req-5ed34136-3f19-428b-bf19-6423b47849c7 None None] MFA Rules not processed for user `2969d23a676c4b7fb6c7b611c5999fa4`. Rule list: `[]` (Enabled: `True`). {{(pid=81582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:21:28.429510 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5ed34136-3f19-428b-bf19-6423b47849c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:21:28.436399 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1292/2578] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:21:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:21:37.308578 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:21:38.421185 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=85,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:21:44.743485 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e6b16-d6c5-4cea-9e33-9aad19dd3fd1 service nova] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:44.744871 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-0d1e6b16-d6c5-4cea-9e33-9aad19dd3fd1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:21:44.800721 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e6b16-d6c5-4cea-9e33-9aad19dd3fd1 service nova] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:44.804404 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9284f-05e9-4c88-860f-1227311b79dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67abf21b99d349dfa7d88a0e18be1fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:44.805007 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9284f-05e9-4c88-860f-1227311b79dc service cinder] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:44.805288 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9284f-05e9-4c88-860f-1227311b79dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:44.805561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9284f-05e9-4c88-860f-1227311b79dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:44.807325 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-14d9284f-05e9-4c88-860f-1227311b79dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:21:44.821016 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9284f-05e9-4c88-860f-1227311b79dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:44.822444 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9284f-05e9-4c88-860f-1227311b79dc service cinder] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:44.833261 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1287/2579] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:21:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:21:45.766583 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed34136-3f19-428b-bf19-6423b47849c7 None None] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:21:45.768019 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-5ed34136-3f19-428b-bf19-6423b47849c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:21:45.827679 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed34136-3f19-428b-bf19-6423b47849c7 None None] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:21:45.831623 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044037a1-9088-4daf-baeb-a72249f66311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:21:45.832241 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044037a1-9088-4daf-baeb-a72249f66311 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:21:45.832519 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044037a1-9088-4daf-baeb-a72249f66311 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:21:45.832791 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044037a1-9088-4daf-baeb-a72249f66311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:21:45.835353 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-044037a1-9088-4daf-baeb-a72249f66311 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:21:45.849874 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044037a1-9088-4daf-baeb-a72249f66311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:21:45.851349 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044037a1-9088-4daf-baeb-a72249f66311 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:21:45.862431 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1293/2580] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:21:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:21:54.828452 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=26 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:21:55.857233 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=26 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:22:00.281750 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9284f-05e9-4c88-860f-1227311b79dc service cinder] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:22:00.292146 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9284f-05e9-4c88-860f-1227311b79dc service cinder] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:22:00.295890 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5ca90-8e1a-42ac-bb4f-24a2ab4c1029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:22:00.296515 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5ca90-8e1a-42ac-bb4f-24a2ab4c1029 service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:22:00.296794 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5ca90-8e1a-42ac-bb4f-24a2ab4c1029 service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:22:00.297066 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5ca90-8e1a-42ac-bb4f-24a2ab4c1029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:22:00.306739 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5ca90-8e1a-42ac-bb4f-24a2ab4c1029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:22:00.308202 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5ca90-8e1a-42ac-bb4f-24a2ab4c1029 service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:22:00.317986 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1288/2581] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:22:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:22:10.313349 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=6 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:22:39.254123 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044037a1-9088-4daf-baeb-a72249f66311 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:22:39.256184 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-044037a1-9088-4daf-baeb-a72249f66311 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:22:39.283647 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044037a1-9088-4daf-baeb-a72249f66311 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:22:39.290583 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d410cc-ce66-4450-9326-d0d2843f3b2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:22:39.291253 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d410cc-ce66-4450-9326-d0d2843f3b2c service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:22:39.291538 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d410cc-ce66-4450-9326-d0d2843f3b2c service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:22:39.291810 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d410cc-ce66-4450-9326-d0d2843f3b2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:22:39.294034 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-98d410cc-ce66-4450-9326-d0d2843f3b2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:22:39.307591 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d410cc-ce66-4450-9326-d0d2843f3b2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:22:39.309188 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d410cc-ce66-4450-9326-d0d2843f3b2c service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:22:39.336152 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1294/2582] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:22:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:22:40.253686 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124fa8f-761e-4d05-88df-cc3607f70826 None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:22:40.254136 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124fa8f-761e-4d05-88df-cc3607f70826 None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:22:40.254254 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d124fa8f-761e-4d05-88df-cc3607f70826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:22:40.274596 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-d124fa8f-761e-4d05-88df-cc3607f70826 None None] MFA Rules not processed for user `4d30a0379df643ff81e1ca01f3e26d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:22:40.275841 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-d124fa8f-761e-4d05-88df-cc3607f70826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:22:40.331083 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1289/2583] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:22:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:22:41.083741 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d410cc-ce66-4450-9326-d0d2843f3b2c service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:22:41.085133 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-98d410cc-ce66-4450-9326-d0d2843f3b2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:22:41.155346 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d410cc-ce66-4450-9326-d0d2843f3b2c service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:22:41.160412 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290977a-5d64-401c-b260-525da810d711 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cc2655cc32e4d63a7000ef41bcbcfea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:22:41.161579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290977a-5d64-401c-b260-525da810d711 service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:22:41.161860 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290977a-5d64-401c-b260-525da810d711 service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:22:41.162137 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290977a-5d64-401c-b260-525da810d711 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:22:41.163912 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-2290977a-5d64-401c-b260-525da810d711 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:22:41.175783 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290977a-5d64-401c-b260-525da810d711 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:22:41.177972 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290977a-5d64-401c-b260-525da810d711 service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:22:41.188017 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1295/2584] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:22:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:22:43.014680 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124fa8f-761e-4d05-88df-cc3607f70826 None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:22:43.025270 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d124fa8f-761e-4d05-88df-cc3607f70826 None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:22:43.028927 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52507d6-f1f4-4b4c-9550-7c2901970e5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:22:43.029543 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52507d6-f1f4-4b4c-9550-7c2901970e5a service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:22:43.029851 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52507d6-f1f4-4b4c-9550-7c2901970e5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:22:43.030185 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52507d6-f1f4-4b4c-9550-7c2901970e5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:22:43.032011 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f52507d6-f1f4-4b4c-9550-7c2901970e5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:22:43.044936 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52507d6-f1f4-4b4c-9550-7c2901970e5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:22:43.046469 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52507d6-f1f4-4b4c-9550-7c2901970e5a service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:22:43.056262 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1290/2585] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:22:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:22:51.183782 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=38 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:22:53.051561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=31 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:22:54.695162 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290977a-5d64-401c-b260-525da810d711 service placement] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:22:54.709628 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290977a-5d64-401c-b260-525da810d711 service placement] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:22:54.715158 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81898e09-426e-4ab6-985d-e0b8c1ee68d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:22:54.715761 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81898e09-426e-4ab6-985d-e0b8c1ee68d4 service nova] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:22:54.716041 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81898e09-426e-4ab6-985d-e0b8c1ee68d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:22:54.716310 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81898e09-426e-4ab6-985d-e0b8c1ee68d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:22:54.718026 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-81898e09-426e-4ab6-985d-e0b8c1ee68d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:22:54.730659 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81898e09-426e-4ab6-985d-e0b8c1ee68d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:22:54.732350 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81898e09-426e-4ab6-985d-e0b8c1ee68d4 service nova] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:22:54.743553 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1296/2586] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:22:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:23:04.738597 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=6 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:24:13.869374 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52507d6-f1f4-4b4c-9550-7c2901970e5a service neutron] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:24:13.878184 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52507d6-f1f4-4b4c-9550-7c2901970e5a service neutron] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:24:13.881871 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f53ebe-a20c-4a2c-8c3c-44dda14c190a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:24:13.882541 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f53ebe-a20c-4a2c-8c3c-44dda14c190a service neutron] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:24:13.882942 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f53ebe-a20c-4a2c-8c3c-44dda14c190a service neutron] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:24:13.883286 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f53ebe-a20c-4a2c-8c3c-44dda14c190a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:24:13.885099 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-02f53ebe-a20c-4a2c-8c3c-44dda14c190a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:24:13.895228 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f53ebe-a20c-4a2c-8c3c-44dda14c190a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:24:13.896766 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f53ebe-a20c-4a2c-8c3c-44dda14c190a service neutron] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:24:13.905068 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1291/2587] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:24:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:24:13.978271 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81898e09-426e-4ab6-985d-e0b8c1ee68d4 service nova] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:24:13.989112 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81898e09-426e-4ab6-985d-e0b8c1ee68d4 service nova] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:24:13.992993 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59277887-3870-4f50-ae0b-11c5656d91c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:24:13.993710 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59277887-3870-4f50-ae0b-11c5656d91c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:24:13.997024 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59277887-3870-4f50-ae0b-11c5656d91c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:24:13.997311 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59277887-3870-4f50-ae0b-11c5656d91c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:24:13.999116 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-59277887-3870-4f50-ae0b-11c5656d91c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:24:14.012797 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59277887-3870-4f50-ae0b-11c5656d91c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:24:14.014209 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59277887-3870-4f50-ae0b-11c5656d91c7 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:24:14.024785 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1297/2588] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:24:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:24:23.901339 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=6 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:24:24.020113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=6 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:24:33.858892 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c95a85-2e85-4413-8d33-c74ecb3ca62e None None] REQUEST_METHOD: `POST` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:24:33.859381 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c95a85-2e85-4413-8d33-c74ecb3ca62e None None] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:24:33.859979 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c95a85-2e85-4413-8d33-c74ecb3ca62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:24:33.879964 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.auth.core [None req-f3c95a85-2e85-4413-8d33-c74ecb3ca62e None None] MFA Rules not processed for user `4d30a0379df643ff81e1ca01f3e26d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:24:33.881150 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-f3c95a85-2e85-4413-8d33-c74ecb3ca62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:24:33.887935 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1292/2589] 10.222.0.27 () {58 vars in 958 bytes} [Mon Oct 17 03:24:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:24:34.197461 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59277887-3870-4f50-ae0b-11c5656d91c7 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:24:34.205925 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59277887-3870-4f50-ae0b-11c5656d91c7 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:24:34.209500 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8d313-f097-4e8b-8e5d-92fb06a461b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:24:34.210090 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8d313-f097-4e8b-8e5d-92fb06a461b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:24:34.210367 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8d313-f097-4e8b-8e5d-92fb06a461b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:24:34.210636 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8d313-f097-4e8b-8e5d-92fb06a461b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:24:34.212336 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-56a8d313-f097-4e8b-8e5d-92fb06a461b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:24:34.223000 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8d313-f097-4e8b-8e5d-92fb06a461b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:24:34.224417 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8d313-f097-4e8b-8e5d-92fb06a461b5 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:24:34.232272 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1298/2590] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:24:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:24:43.874061 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=3 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:24:44.228670 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=6 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:25:47.197913 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c95a85-2e85-4413-8d33-c74ecb3ca62e None None] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:25:47.210779 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c95a85-2e85-4413-8d33-c74ecb3ca62e None None] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:25:47.215870 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbdb26-66a9-4d6a-ab77-fbc3e2d4bc60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cc2655cc32e4d63a7000ef41bcbcfea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:25:47.216510 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbdb26-66a9-4d6a-ab77-fbc3e2d4bc60 service placement] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:25:47.216793 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbdb26-66a9-4d6a-ab77-fbc3e2d4bc60 service placement] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:25:47.217066 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbdb26-66a9-4d6a-ab77-fbc3e2d4bc60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:25:47.218992 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c9bbdb26-66a9-4d6a-ab77-fbc3e2d4bc60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:25:47.231977 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbdb26-66a9-4d6a-ab77-fbc3e2d4bc60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:25:47.233573 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bbdb26-66a9-4d6a-ab77-fbc3e2d4bc60 service placement] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:25:47.243648 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1293/2591] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:25:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:25:51.321952 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8d313-f097-4e8b-8e5d-92fb06a461b5 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:25:51.323437 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-56a8d313-f097-4e8b-8e5d-92fb06a461b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:25:51.339184 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8d313-f097-4e8b-8e5d-92fb06a461b5 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:25:51.345155 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b33c20-e48e-4c73-b681-a0b64f348681 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cc2655cc32e4d63a7000ef41bcbcfea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:25:51.345924 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b33c20-e48e-4c73-b681-a0b64f348681 service placement] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:25:51.346207 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b33c20-e48e-4c73-b681-a0b64f348681 service placement] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:25:51.346511 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b33c20-e48e-4c73-b681-a0b64f348681 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:25:51.349243 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-70b33c20-e48e-4c73-b681-a0b64f348681 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:25:51.361139 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b33c20-e48e-4c73-b681-a0b64f348681 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:25:51.363649 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b33c20-e48e-4c73-b681-a0b64f348681 service placement] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:25:51.374474 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1299/2592] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:25:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:25:54.432421 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbdb26-66a9-4d6a-ab77-fbc3e2d4bc60 service placement] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:25:54.445407 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bbdb26-66a9-4d6a-ab77-fbc3e2d4bc60 service placement] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:25:54.450717 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b940a-39d2-4711-b7ba-6feb6743b9cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cc2655cc32e4d63a7000ef41bcbcfea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:25:54.451521 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b940a-39d2-4711-b7ba-6feb6743b9cd service placement] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:25:54.451797 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b940a-39d2-4711-b7ba-6feb6743b9cd service placement] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:25:54.452069 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b940a-39d2-4711-b7ba-6feb6743b9cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:25:54.453827 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-257b940a-39d2-4711-b7ba-6feb6743b9cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:25:54.472780 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b940a-39d2-4711-b7ba-6feb6743b9cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:25:54.474397 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b940a-39d2-4711-b7ba-6feb6743b9cd service placement] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:25:54.491479 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1294/2593] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:25:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:25:54.497507 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b33c20-e48e-4c73-b681-a0b64f348681 service placement] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:25:54.569657 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b33c20-e48e-4c73-b681-a0b64f348681 service placement] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:25:54.574239 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8cdbe-9438-4ab6-a5be-a19dde251981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:25:54.575012 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8cdbe-9438-4ab6-a5be-a19dde251981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:25:54.575367 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8cdbe-9438-4ab6-a5be-a19dde251981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:25:54.575651 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8cdbe-9438-4ab6-a5be-a19dde251981 admin admin] PATH_INFO: `/v3/users/2969d23a676c4b7fb6c7b611c5999fa4` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:25:54.576790 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8cdbe-9438-4ab6-a5be-a19dde251981 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2969d23a676c4b7fb6c7b611c5999fa4)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:25:54.578729 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8cdbe-9438-4ab6-a5be-a19dde251981 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:25:54.629048 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4ee8cdbe-9438-4ab6-a5be-a19dde251981 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2969d23a676c4b7fb6c7b611c5999fa4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:54.637383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4ee8cdbe-9438-4ab6-a5be-a19dde251981 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2969d23a676c4b7fb6c7b611c5999fa4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:54.638075 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4ee8cdbe-9438-4ab6-a5be-a19dde251981 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2969d23a676c4b7fb6c7b611c5999fa4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:54.644807 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4ee8cdbe-9438-4ab6-a5be-a19dde251981 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2969d23a676c4b7fb6c7b611c5999fa4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:54.656838 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-4ee8cdbe-9438-4ab6-a5be-a19dde251981 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2969d23a676c4b7fb6c7b611c5999fa4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:54.663182 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1300/2594] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:25:54 2022] DELETE /identity/v3/users/2969d23a676c4b7fb6c7b611c5999fa4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:25:54.833865 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b940a-39d2-4711-b7ba-6feb6743b9cd service placement] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:25:54.845220 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b940a-39d2-4711-b7ba-6feb6743b9cd service placement] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:25:54.902425 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbc5b6-d851-4706-8a47-7a26cae3450f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:25:54.902989 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbc5b6-d851-4706-8a47-7a26cae3450f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:25:54.903273 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbc5b6-d851-4706-8a47-7a26cae3450f admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:25:54.903547 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbc5b6-d851-4706-8a47-7a26cae3450f admin admin] PATH_INFO: `/v3/projects/f7a92b100ac14d44aff5e4083a49f5ad` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:25:54.904660 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbc5b6-d851-4706-8a47-7a26cae3450f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7a92b100ac14d44aff5e4083a49f5ad)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:25:54.906556 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbc5b6-d851-4706-8a47-7a26cae3450f admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:25:54.951643 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-11fbc5b6-d851-4706-8a47-7a26cae3450f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7a92b100ac14d44aff5e4083a49f5ad', 'request_id': 'req-11fbc5b6-d851-4706-8a47-7a26cae3450f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:54.957134 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-11fbc5b6-d851-4706-8a47-7a26cae3450f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7a92b100ac14d44aff5e4083a49f5ad', 'request_id': 'req-11fbc5b6-d851-4706-8a47-7a26cae3450f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:54.963244 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-11fbc5b6-d851-4706-8a47-7a26cae3450f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7a92b100ac14d44aff5e4083a49f5ad', 'request_id': 'req-11fbc5b6-d851-4706-8a47-7a26cae3450f'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:54.976060 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-11fbc5b6-d851-4706-8a47-7a26cae3450f admin admin] The token cache is being invalidate because project f7a92b100ac14d44aff5e4083a49f5ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:25:54.976373 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-11fbc5b6-d851-4706-8a47-7a26cae3450f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:54.977404 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-11fbc5b6-d851-4706-8a47-7a26cae3450f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:54.978988 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1295/2595] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:25:54 2022] DELETE /identity/v3/projects/f7a92b100ac14d44aff5e4083a49f5ad => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:25:54.982916 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8cdbe-9438-4ab6-a5be-a19dde251981 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:25:54.984368 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-4ee8cdbe-9438-4ab6-a5be-a19dde251981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:25:54.998455 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8cdbe-9438-4ab6-a5be-a19dde251981 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:25:55.058176 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f60677-99c7-4fac-834b-9a0b7f7f4d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:25:55.058821 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f60677-99c7-4fac-834b-9a0b7f7f4d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:25:55.059161 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f60677-99c7-4fac-834b-9a0b7f7f4d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:25:55.059436 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f60677-99c7-4fac-834b-9a0b7f7f4d62 admin admin] PATH_INFO: `/v3/users/c8ca1a9883fd4b0f9cb82e54fcc89e50` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:25:55.060579 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f60677-99c7-4fac-834b-9a0b7f7f4d62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8ca1a9883fd4b0f9cb82e54fcc89e50)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:25:55.062505 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f60677-99c7-4fac-834b-9a0b7f7f4d62 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:25:55.068113 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:25:55.111135 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-97f60677-99c7-4fac-834b-9a0b7f7f4d62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8ca1a9883fd4b0f9cb82e54fcc89e50'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:55.119580 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-97f60677-99c7-4fac-834b-9a0b7f7f4d62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8ca1a9883fd4b0f9cb82e54fcc89e50'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:55.121338 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-97f60677-99c7-4fac-834b-9a0b7f7f4d62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8ca1a9883fd4b0f9cb82e54fcc89e50'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:55.129863 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-97f60677-99c7-4fac-834b-9a0b7f7f4d62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8ca1a9883fd4b0f9cb82e54fcc89e50'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:55.142858 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-97f60677-99c7-4fac-834b-9a0b7f7f4d62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8ca1a9883fd4b0f9cb82e54fcc89e50'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:55.149409 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1301/2596] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:25:54 2022] DELETE /identity/v3/users/c8ca1a9883fd4b0f9cb82e54fcc89e50 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:25:55.332478 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbc5b6-d851-4706-8a47-7a26cae3450f admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:25:55.334072 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-11fbc5b6-d851-4706-8a47-7a26cae3450f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:25:55.348237 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbc5b6-d851-4706-8a47-7a26cae3450f admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:25:55.411572 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096c45c-3bf9-4c54-9313-7ce1b4645e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:25:55.412387 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096c45c-3bf9-4c54-9313-7ce1b4645e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:25:55.412667 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096c45c-3bf9-4c54-9313-7ce1b4645e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:25:55.412940 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096c45c-3bf9-4c54-9313-7ce1b4645e31 admin admin] PATH_INFO: `/v3/projects/53e87d643123424b964b82af7aaec9ba` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:25:55.414077 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096c45c-3bf9-4c54-9313-7ce1b4645e31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53e87d643123424b964b82af7aaec9ba)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:25:55.416031 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096c45c-3bf9-4c54-9313-7ce1b4645e31 admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:25:55.422726 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:25:55.467561 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c096c45c-3bf9-4c54-9313-7ce1b4645e31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53e87d643123424b964b82af7aaec9ba', 'request_id': 'req-c096c45c-3bf9-4c54-9313-7ce1b4645e31'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:55.474141 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c096c45c-3bf9-4c54-9313-7ce1b4645e31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53e87d643123424b964b82af7aaec9ba', 'request_id': 'req-c096c45c-3bf9-4c54-9313-7ce1b4645e31'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:55.480618 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c096c45c-3bf9-4c54-9313-7ce1b4645e31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53e87d643123424b964b82af7aaec9ba', 'request_id': 'req-c096c45c-3bf9-4c54-9313-7ce1b4645e31'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:55.493134 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c096c45c-3bf9-4c54-9313-7ce1b4645e31 admin admin] The token cache is being invalidate because project 53e87d643123424b964b82af7aaec9ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81581) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:25:55.493435 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c096c45c-3bf9-4c54-9313-7ce1b4645e31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:55.494090 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-c096c45c-3bf9-4c54-9313-7ce1b4645e31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:25:55.496600 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1296/2597] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:25:55 2022] DELETE /identity/v3/projects/53e87d643123424b964b82af7aaec9ba => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:26:05.145756 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:26:05.490027 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:26:22.076964 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f60677-99c7-4fac-834b-9a0b7f7f4d62 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:26:22.078390 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-97f60677-99c7-4fac-834b-9a0b7f7f4d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:26:22.134871 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f60677-99c7-4fac-834b-9a0b7f7f4d62 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:26:22.181566 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b85d9-7dbf-4779-a7da-a26bc8e9f399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:26:22.182148 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b85d9-7dbf-4779-a7da-a26bc8e9f399 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:26:22.182428 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b85d9-7dbf-4779-a7da-a26bc8e9f399 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:26:22.182704 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b85d9-7dbf-4779-a7da-a26bc8e9f399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:26:22.184454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8b7b85d9-7dbf-4779-a7da-a26bc8e9f399 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:26:22.241071 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b85d9-7dbf-4779-a7da-a26bc8e9f399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:26:22.242497 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b85d9-7dbf-4779-a7da-a26bc8e9f399 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:26:22.251375 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1302/2598] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:26:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:26:23.097290 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096c45c-3bf9-4c54-9313-7ce1b4645e31 admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:26:23.099486 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-c096c45c-3bf9-4c54-9313-7ce1b4645e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:26:23.183101 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096c45c-3bf9-4c54-9313-7ce1b4645e31 admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:26:23.188860 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2361009-9ac2-4683-94f2-e381e940d18e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01637fc07ba846aebcb42325bed46216', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:26:23.189811 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2361009-9ac2-4683-94f2-e381e940d18e service nova] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:26:23.190349 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2361009-9ac2-4683-94f2-e381e940d18e service nova] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:26:23.190998 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2361009-9ac2-4683-94f2-e381e940d18e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:26:23.193977 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e2361009-9ac2-4683-94f2-e381e940d18e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:26:23.209436 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2361009-9ac2-4683-94f2-e381e940d18e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:26:23.211553 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2361009-9ac2-4683-94f2-e381e940d18e service nova] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:26:23.239122 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1297/2599] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:26:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:26:32.247319 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=68 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:26:33.219298 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=34 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:26:35.654522 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b85d9-7dbf-4779-a7da-a26bc8e9f399 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:26:35.656175 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-8b7b85d9-7dbf-4779-a7da-a26bc8e9f399 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:26:35.667814 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b85d9-7dbf-4779-a7da-a26bc8e9f399 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:26:35.671447 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c427c9-5724-441a-84d3-09891e1d0335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526933e6ad004493b071f5da5caa09a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:26:35.672050 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c427c9-5724-441a-84d3-09891e1d0335 service neutron] REQUEST_METHOD: `GET` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:26:35.672334 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c427c9-5724-441a-84d3-09891e1d0335 service neutron] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:26:35.672610 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c427c9-5724-441a-84d3-09891e1d0335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:26:35.674316 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-98c427c9-5724-441a-84d3-09891e1d0335 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:26:35.740519 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c427c9-5724-441a-84d3-09891e1d0335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:26:35.741951 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c427c9-5724-441a-84d3-09891e1d0335 service neutron] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:26:35.751291 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1303/2600] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:26:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:26:42.027070 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2361009-9ac2-4683-94f2-e381e940d18e service nova] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:26:42.028657 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-e2361009-9ac2-4683-94f2-e381e940d18e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:26:42.044154 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2361009-9ac2-4683-94f2-e381e940d18e service nova] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:26:42.048847 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c9a3f-39e6-497a-99d8-3abc5e43581b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:26:42.049471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c9a3f-39e6-497a-99d8-3abc5e43581b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:26:42.049751 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c9a3f-39e6-497a-99d8-3abc5e43581b admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:26:42.050119 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c9a3f-39e6-497a-99d8-3abc5e43581b admin admin] PATH_INFO: `/v3/users/4d30a0379df643ff81e1ca01f3e26d3d` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:26:42.051496 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c9a3f-39e6-497a-99d8-3abc5e43581b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d30a0379df643ff81e1ca01f3e26d3d)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:26:42.053471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c9a3f-39e6-497a-99d8-3abc5e43581b admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:26:42.100793 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4b5c9a3f-39e6-497a-99d8-3abc5e43581b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d30a0379df643ff81e1ca01f3e26d3d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.107931 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4b5c9a3f-39e6-497a-99d8-3abc5e43581b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d30a0379df643ff81e1ca01f3e26d3d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.108636 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4b5c9a3f-39e6-497a-99d8-3abc5e43581b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d30a0379df643ff81e1ca01f3e26d3d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.115966 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4b5c9a3f-39e6-497a-99d8-3abc5e43581b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d30a0379df643ff81e1ca01f3e26d3d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.128087 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-4b5c9a3f-39e6-497a-99d8-3abc5e43581b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d30a0379df643ff81e1ca01f3e26d3d'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.134473 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1298/2601] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:26:42 2022] DELETE /identity/v3/users/4d30a0379df643ff81e1ca01f3e26d3d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:26:42.311335 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c427c9-5724-441a-84d3-09891e1d0335 service neutron] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:26:42.322226 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c427c9-5724-441a-84d3-09891e1d0335 service neutron] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:26:42.380883 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b77f81-86f1-49e5-90c1-f2a813460283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:26:42.381543 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b77f81-86f1-49e5-90c1-f2a813460283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:26:42.381822 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b77f81-86f1-49e5-90c1-f2a813460283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:26:42.382096 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b77f81-86f1-49e5-90c1-f2a813460283 admin admin] PATH_INFO: `/v3/projects/917f1029c33f4607b398287532f2fb4b` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:26:42.383250 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b77f81-86f1-49e5-90c1-f2a813460283 admin admin] RBAC: Authorizing `identity:delete_project(project_id=917f1029c33f4607b398287532f2fb4b)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:26:42.385199 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b77f81-86f1-49e5-90c1-f2a813460283 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:26:42.432352 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c3b77f81-86f1-49e5-90c1-f2a813460283 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '917f1029c33f4607b398287532f2fb4b', 'request_id': 'req-c3b77f81-86f1-49e5-90c1-f2a813460283'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.437733 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c3b77f81-86f1-49e5-90c1-f2a813460283 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '917f1029c33f4607b398287532f2fb4b', 'request_id': 'req-c3b77f81-86f1-49e5-90c1-f2a813460283'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.444383 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c3b77f81-86f1-49e5-90c1-f2a813460283 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '917f1029c33f4607b398287532f2fb4b', 'request_id': 'req-c3b77f81-86f1-49e5-90c1-f2a813460283'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.457122 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c3b77f81-86f1-49e5-90c1-f2a813460283 admin admin] The token cache is being invalidate because project 917f1029c33f4607b398287532f2fb4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:26:42.457425 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c3b77f81-86f1-49e5-90c1-f2a813460283 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.458057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-c3b77f81-86f1-49e5-90c1-f2a813460283 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.459536 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1304/2602] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:26:42 2022] DELETE /identity/v3/projects/917f1029c33f4607b398287532f2fb4b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:26:42.463930 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c9a3f-39e6-497a-99d8-3abc5e43581b admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:26:42.465436 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-4b5c9a3f-39e6-497a-99d8-3abc5e43581b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:26:42.478689 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c9a3f-39e6-497a-99d8-3abc5e43581b admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:26:42.533992 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01b684-8a94-4c39-9b5d-feb770869b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:26:42.534614 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01b684-8a94-4c39-9b5d-feb770869b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:26:42.534912 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01b684-8a94-4c39-9b5d-feb770869b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:26:42.535189 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01b684-8a94-4c39-9b5d-feb770869b0e admin admin] PATH_INFO: `/v3/users/b86d602523364d58bcd0d5df3aadb8eb` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:26:42.536576 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b01b684-8a94-4c39-9b5d-feb770869b0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b86d602523364d58bcd0d5df3aadb8eb)` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:26:42.538471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b01b684-8a94-4c39-9b5d-feb770869b0e admin admin] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:26:42.583164 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3b01b684-8a94-4c39-9b5d-feb770869b0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b86d602523364d58bcd0d5df3aadb8eb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.589807 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3b01b684-8a94-4c39-9b5d-feb770869b0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b86d602523364d58bcd0d5df3aadb8eb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.590475 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3b01b684-8a94-4c39-9b5d-feb770869b0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b86d602523364d58bcd0d5df3aadb8eb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.597484 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3b01b684-8a94-4c39-9b5d-feb770869b0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b86d602523364d58bcd0d5df3aadb8eb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.605174 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:26:42.610731 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.notifications [None req-3b01b684-8a94-4c39-9b5d-feb770869b0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b86d602523364d58bcd0d5df3aadb8eb'} {{(pid=81581) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.617729 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1299/2603] 10.222.0.27 () {58 vars in 1224 bytes} [Mon Oct 17 03:26:42 2022] DELETE /identity/v3/users/b86d602523364d58bcd0d5df3aadb8eb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:26:42.789057 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b77f81-86f1-49e5-90c1-f2a813460283 admin admin] Authenticating user token {{(pid=81582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:26:42.791114 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.fernet_utils [None req-c3b77f81-86f1-49e5-90c1-f2a813460283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:26:42.805255 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b77f81-86f1-49e5-90c1-f2a813460283 admin admin] Validating token access rules against request {{(pid=81582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:26:42.825925 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:26:42.863631 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfb5247-a6be-4969-9352-4134d2730ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84d693b9d0643c69861a5541952b43b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb6be797d9e4443fad02ceee69bd83a7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:26:42.864942 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb5247-a6be-4969-9352-4134d2730ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:26:42.865229 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb5247-a6be-4969-9352-4134d2730ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:26:42.865508 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfb5247-a6be-4969-9352-4134d2730ad4 admin admin] PATH_INFO: `/v3/projects/bd52b177a16a4ea1b2dff23d32ab4673` {{(pid=81582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:26:42.866676 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfb5247-a6be-4969-9352-4134d2730ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd52b177a16a4ea1b2dff23d32ab4673)` {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:26:42.868645 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfb5247-a6be-4969-9352-4134d2730ad4 admin admin] RBAC: Authorization granted {{(pid=81582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:26:42.914052 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7bfb5247-a6be-4969-9352-4134d2730ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd52b177a16a4ea1b2dff23d32ab4673', 'request_id': 'req-7bfb5247-a6be-4969-9352-4134d2730ad4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.919357 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7bfb5247-a6be-4969-9352-4134d2730ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd52b177a16a4ea1b2dff23d32ab4673', 'request_id': 'req-7bfb5247-a6be-4969-9352-4134d2730ad4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.925445 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7bfb5247-a6be-4969-9352-4134d2730ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd52b177a16a4ea1b2dff23d32ab4673', 'request_id': 'req-7bfb5247-a6be-4969-9352-4134d2730ad4'} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.938495 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7bfb5247-a6be-4969-9352-4134d2730ad4 admin admin] The token cache is being invalidate because project bd52b177a16a4ea1b2dff23d32ab4673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81582) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:26:42.938779 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7bfb5247-a6be-4969-9352-4134d2730ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.939454 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG keystone.notifications [None req-7bfb5247-a6be-4969-9352-4134d2730ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81582) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:26:42.940876 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: [pid: 81582|app: 0|req: 1305/2604] 10.222.0.27 () {58 vars in 1233 bytes} [Mon Oct 17 03:26:42 2022] DELETE /identity/v3/projects/bd52b177a16a4ea1b2dff23d32ab4673 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:26:52.616898 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:26:52.935602 dv-cinder-846977-2-iscsi devstack@keystone.service[81582]: DEBUG dbcounter [-] [81582] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:02.310471 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01b684-8a94-4c39-9b5d-feb770869b0e admin admin] Authenticating user token {{(pid=81581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:02.311992 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-3b01b684-8a94-4c39-9b5d-feb770869b0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:31:02.457196 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01b684-8a94-4c39-9b5d-feb770869b0e admin admin] Validating token access rules against request {{(pid=81581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:02.519109 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d65927-1a65-46b3-bb90-bd7810bb5ff1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cc2655cc32e4d63a7000ef41bcbcfea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '89b0ee95955f465eb9ee5b8ae337d9c2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:02.519762 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d65927-1a65-46b3-bb90-bd7810bb5ff1 service placement] REQUEST_METHOD: `GET` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:02.520040 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d65927-1a65-46b3-bb90-bd7810bb5ff1 service placement] SCRIPT_NAME: `/identity` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:02.520314 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d65927-1a65-46b3-bb90-bd7810bb5ff1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:02.522291 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.fernet_utils [None req-88d65927-1a65-46b3-bb90-bd7810bb5ff1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:31:02.534233 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d65927-1a65-46b3-bb90-bd7810bb5ff1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:02.535826 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d65927-1a65-46b3-bb90-bd7810bb5ff1 service placement] RBAC: Authorization granted {{(pid=81581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:02.546591 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: [pid: 81581|app: 0|req: 1300/2605] 10.222.0.27 () {60 vars in 1365 bytes} [Mon Oct 17 03:31:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:31:12.542183 dv-cinder-846977-2-iscsi devstack@keystone.service[81581]: DEBUG dbcounter [-] [81581] Writing DB stats keystone:SELECT=89 {{(pid=81581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}